-- Logs begin at Wed 2022-08-31 18:25:31 UTC, end at Wed 2022-08-31 20:27:28 UTC. -- Aug 31 19:04:06.801194 dv-cinder-852209-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 31 19:04:06.811863 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 31 19:04:06.813892 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 31 19:04:06 2022] *** Aug 31 19:04:06.813892 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 31 19:04:06.813892 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 31 19:04:06.813892 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: nodename: dv-cinder-852209-9-iscsi Aug 31 19:04:06.813892 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: machine: x86_64 Aug 31 19:04:06.813892 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: clock source: unix Aug 31 19:04:06.813892 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: pcre jit disabled Aug 31 19:04:06.813892 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: detected number of CPU cores: 4 Aug 31 19:04:06.813892 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: current working directory: / Aug 31 19:04:06.813892 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: detected binary path: /usr/bin/uwsgi-core Aug 31 19:04:06.813892 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: your processes number limit is 47680 Aug 31 19:04:06.814528 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: your memory page size is 4096 bytes Aug 31 19:04:06.814528 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: detected max file descriptor number: 2048 Aug 31 19:04:06.814528 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: lock engine: pthread robust mutexes Aug 31 19:04:06.814528 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: thunder lock: enabled Aug 31 19:04:06.814528 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 31 19:04:06.814528 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 31 19:04:06.855196 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: Python main interpreter initialized at 0x560947abe9c0 Aug 31 19:04:06.855196 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: python threads support enabled Aug 31 19:04:06.855196 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: your server socket listen backlog is limited to 100 connections Aug 31 19:04:06.855196 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: your mercy for graceful operations on workers is 90 seconds Aug 31 19:04:06.855557 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: mapped 403077 bytes (393 KB) for 2 cores Aug 31 19:04:06.855632 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: *** Operational MODE: preforking *** Aug 31 19:04:06.855898 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: *** uWSGI is running in multiple interpreter mode *** Aug 31 19:04:06.855898 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: spawned uWSGI master process (pid: 81856) Aug 31 19:04:06.856131 dv-cinder-852209-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 31 19:04:06.861673 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: spawned uWSGI worker 1 (pid: 81868, cores: 1) Aug 31 19:04:06.861673 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: spawned uWSGI worker 2 (pid: 81869, cores: 1) Aug 31 19:04:06.861673 dv-cinder-852209-9-iscsi devstack@keystone.service[81856]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 31 19:04:11.211404 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 19:04:11.211404 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: from cryptography.utils import int_from_bytes Aug 31 19:04:11.212373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 19:04:11.212373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: from cryptography.utils import int_from_bytes Aug 31 19:04:11.213040 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 19:04:11.213040 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: from cryptography.utils import int_from_bytes Aug 31 19:04:11.213555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 19:04:11.213555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: from cryptography.utils import int_from_bytes Aug 31 19:04:11.262792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 19:04:11.262992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 19:04:11.264945 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] command line args: [] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 19:04:11.264945 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 19:04:11.264945 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ================================================================================ {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 19:04:11.264945 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] admin_token = **** {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.264945 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] config_dir = [] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.265267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 19:04:11.265337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.265397 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 19:04:11.265456 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] config_source = [] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.265521 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] command line args: [] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 19:04:11.265581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] debug = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.265644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 19:04:11.265708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.266146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ================================================================================ {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 19:04:11.266225 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.266291 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] admin_token = **** {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.266357 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.266422 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] config_dir = [] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.266422 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.266543 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.266609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] config_source = [] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.266674 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.266739 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] debug = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.266804 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] list_limit = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.267027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.267147 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] log_config_append = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.267216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.267281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.267345 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.267415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] log_dir = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.267415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] log_file = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.267533 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.267593 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.267652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.267716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.267775 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] list_limit = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.267775 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] log_config_append = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.267886 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.267946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.268011 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.268075 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] log_dir = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.268134 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.268204 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] log_file = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.268269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.268330 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.268387 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.268445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.268508 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.268575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.268640 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.268887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.269034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.269100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.269168 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.269233 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.269299 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.269359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.269551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.269628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.269760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.272290 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.272290 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.272645 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.272774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.272774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.273005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.59/identity {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.273123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.273241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] publish_errors = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.273241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.273464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.273584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.273696 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.273815 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.273815 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.274040 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.59/identity {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.274146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.274242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] publish_errors = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.274337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] standard_threads = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.274437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.274607 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.274730 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.274850 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.274980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.275106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.275106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] use_journal = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.275326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.275443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] use_json = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.275561 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.275674 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] use_stderr = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.275796 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] standard_threads = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.275909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] use_syslog = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.276484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.276567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.276640 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.276725 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.276798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.276873 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.276950 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] use_journal = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.276950 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] use_json = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.277089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.277168 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] use_stderr = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.277239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.277364 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] use_syslog = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.277364 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:04:11.277509 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.277581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.277732 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.277806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.277882 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.277954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.278030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.278101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.278337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.278419 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.278497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.278569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.278648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.278719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.278797 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.278895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.278969 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.279040 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.279110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.279181 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.279267 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.279346 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.279422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.279495 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.279570 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.279712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.279796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.279869 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.279939 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.280008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.280078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.280078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.280211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.280288 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.280359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.280428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.280497 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.280572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.280651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.280722 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.280798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.280870 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.280939 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.281014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.281096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.281169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.281303 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.281420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.281533 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.281645 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.281834 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.281946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.282043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.282138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.282232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.282339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.282449 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.282545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.282640 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.282734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.282828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.282965 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.283064 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.283165 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.283261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.283362 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.283458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.283560 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.283656 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.283762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.283857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.283952 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.284046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.284149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.284149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.284343 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.284444 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.284538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.284633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.284826 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.284926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.285021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.285116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.285243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.285243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.285539 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.285539 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.285728 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.285829 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.285932 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.286030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.286160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.286282 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.286395 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.286510 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.286622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.286740 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.286863 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.287025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.287149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.287263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.287382 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.287502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.287623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.287745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.287860 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.287979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.288092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.288211 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.288325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.288325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.288546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.288661 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.288773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.288885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.288998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.289119 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.289242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.289356 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.289507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.289674 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.289794 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.289920 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.290036 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.290155 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.290269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.290390 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.290516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.290630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.290753 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.290882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.290882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.291082 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.291183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.291314 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.291432 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.291539 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.291539 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.291716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.291813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.291917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.292015 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.292111 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.292191 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.292267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.292346 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.292492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.292573 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.292646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.292646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.292773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.292844 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.292923 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.293003 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.293075 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.293152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.293229 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.293229 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.293371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.293454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.293535 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.293608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.293684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.293684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.293821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.293893 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.293973 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.294128 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.294218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.294366 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.294445 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.294605 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.294692 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.294850 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.295001 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.297627 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.297757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.297882 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.297978 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.298054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.298126 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.298198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.298269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.298340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.298420 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.298498 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.298571 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.298642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.298713 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.298784 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.298862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.298957 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.299028 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.299109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.299221 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.299343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.299425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.299425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.299564 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.299636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.299707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.299778 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.299907 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.299985 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.300057 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.300129 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.300200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] auth.external = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.300270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.300349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.300448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.300536 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.300611 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.300682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.300754 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.300826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] auth.password = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.300905 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] auth.external = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.300979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] auth.token = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.301051 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.301123 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.301123 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.301253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.301253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.301394 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.301466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] auth.password = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.301601 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.301680 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] auth.token = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.301751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.301831 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.301905 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.301976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.302047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.302047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] credential.caching = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.302174 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.302174 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.302320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.302392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.302464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.302534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.302605 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.302684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.302757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.302828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] credential.caching = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.302945 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.303030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.303104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.303282 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.303366 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.303532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.303612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.303770 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.303849 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.304011 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.304092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.304249 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.304331 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.304500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.304613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.304751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.304832 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.304990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.305639 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.305743 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.305819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.306033 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.306117 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] federation.caching = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.306277 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.306355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.306529 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.306608 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.306781 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.306970 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.307053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.307242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.307324 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.307484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] federation.caching = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.307582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.307723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.307804 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.307964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.308077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.308209 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.308323 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.308425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.308524 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.308676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.308782 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.308896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] identity.caching = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.309005 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.309144 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.309248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.309378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.309486 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.309617 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.309734 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.309839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.309942 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.310097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.310209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.310324 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] identity.caching = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.310431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.310570 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.310683 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.310819 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.311063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.311188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.311293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.311392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.311498 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.311598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.311706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.311815 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.311917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.312016 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.312115 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.312214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.312315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.312460 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.312574 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.312705 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.312818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.312926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.313034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.313154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.313265 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.313418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.313760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.313760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.313896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.313979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.314053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.314125 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.314197 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.314307 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.314387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.314546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.314627 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.314789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.314888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.315051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.315142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.315302 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.315383 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.315534 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.315613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.315763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.315841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.316000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.316081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.316241 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.316320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.316481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.316560 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.316733 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.316876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.316952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.317024 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.317196 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.317289 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.317423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.317506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.317664 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.317745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.317900 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.317982 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.318145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.318227 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.318376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.318459 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.318612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.318692 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.318840 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.318937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.319105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.319196 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.319342 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.319423 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.319575 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.319658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.319815 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.319937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.320086 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.320166 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.320287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.320369 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.320525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.320606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.321452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.321549 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.321637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.321713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.321854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.321945 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.322108 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.322190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.322348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.322432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.322589 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.322672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.322828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.322959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.323098 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.323188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.323339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.323427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.323574 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.323665 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.323813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.323896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.324050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.324133 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.324295 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.324619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.324712 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.324789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.324862 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.324932 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.324996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.325084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.325223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.325307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.325471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.325554 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.325707 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.325796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.325937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.326018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.326183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.326264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.326424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.326513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.326663 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.326745 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.326927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.327018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.327167 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.327262 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.327398 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.327480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.327646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.327728 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.327883 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.327972 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.328119 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.328200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.328356 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.328440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.328593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.328684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.328829 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.328910 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.329069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.329151 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.329303 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.329460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.329540 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.329613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.329777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.329859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.330219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.330321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.330321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.330448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.330528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.330603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] resource.caching = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.330679 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.330762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.330967 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.331073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.331260 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.331359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.331456 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.337772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.337863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.337928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.338003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.338085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] resource.caching = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.338160 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.338237 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.338322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.338397 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.338471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.338549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.338624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] role.caching = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.338707 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.338783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] role.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.338846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.338954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.339030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.339105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.339192 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.339269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.339344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.339418 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.339491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.339564 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.339646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] role.caching = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.339711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.339773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] role.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.339834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.339907 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.339980 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.340066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.340132 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.340206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.340280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.340353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.340435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.340511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.340584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.340658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.340731 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.340805 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.340805 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.340948 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.341022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.341022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.341155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.341233 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.341316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.341391 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.341471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.341547 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.341621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.341694 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.341775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.341775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.341931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.342011 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.342075 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.342148 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.342231 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.342307 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.342381 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.342455 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.342528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.342602 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.342683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.342759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.342833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.342964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.343042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.343116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.343199 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.343277 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.343351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.343425 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.343499 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.343622 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.343713 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.343790 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.343863 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.343937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.344011 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.344092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] token.caching = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.344168 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.344242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.344315 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.344389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.344462 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.344544 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.344620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.344694 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.344767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.344840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.344913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] token.caching = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.344995 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.345070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.345144 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.345218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.345291 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.345365 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.345446 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.345522 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.345595 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.345669 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.345742 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.345815 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.345896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.345971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.346045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.346118 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.346198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.346273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.346354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.346437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.346523 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.346599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.346673 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.346746 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.346819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.connection = **** {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.346928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.347032 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.347114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.347192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.347268 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.347352 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.347436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.347513 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.347594 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.347670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.347744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.connection = **** {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.347817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.347890 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.347963 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.347963 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.348107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.348181 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.348253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.348327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.348400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.348400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.348630 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.348705 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.348779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.348852 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.348925 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.349007 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.349083 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.349156 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.349229 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.349302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.349375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.349500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.349586 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.349733 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.349820 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.349895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81869) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 19:04:11.349970 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.350202 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.350446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.350678 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.350946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.351184 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:04:11.351355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81868) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 19:04:11.384895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.385071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.385147 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.385225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.385340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.385450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.393842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.394160 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.427046 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.427201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.427295 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.427376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.518061 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.519097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.572288 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.572519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.572839 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.573124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.738275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.738464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.738549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.738654 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.738737 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.738815 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.738963 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.739051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.739195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.739281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.739417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.739500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.763675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.763892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.763992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.764116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.764205 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.764337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.764433 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.764560 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.764646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.764777 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.764860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.764987 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.765079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.765201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.776274 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.776434 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.796717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.796913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.796999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.797095 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.797180 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.797245 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.797377 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.797463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.797598 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.797681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.797814 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.797896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.798029 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81869) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.798113 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81868) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:04:11.815447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.815657 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 19:04:11.815741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.815899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 19:04:11.816238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.816321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.816440 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 19:04:11.816522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 19:04:11.816711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.816835 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.816981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.817109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.817242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.817318 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.817499 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.817593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.826753 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.827064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.827250 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.827338 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.827538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.827626 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.827815 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.828103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.834430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.835073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.837834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.838114 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.838398 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.838490 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.838754 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.838844 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.839077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.839186 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.839336 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.839418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.839640 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.839722 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.839910 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.839991 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.840189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.840283 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.840472 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.840553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.840750 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.840830 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.841031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.841303 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.864141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.864424 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.865857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.866138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.866226 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.866443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.870910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.871104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.871193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.871395 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.871481 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.871687 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.877741 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.878024 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.878173 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.878466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.882073 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.882369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.882460 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.882698 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.882798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.883002 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.883098 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.883330 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.883415 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.883663 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.883756 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.883950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.884222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.885742 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.886041 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.886336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.902282 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.902467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.902560 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.902753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.906485 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.906665 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.909597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.909821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.909909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.911106 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.914143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.915017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.918008 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.918140 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.921151 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.921324 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.921461 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.921649 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.921735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.921947 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.922034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.922243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.922328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.922534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.922641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.922948 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.938446 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.938660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.938759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.938980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.939068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.939260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.939345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.939537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.946252 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.946371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.948048 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.948154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.949354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.949574 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.949662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.949858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.952379 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.952540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.952664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.952835 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.952958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.953137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.953242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.953429 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.953522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.953735 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.965586 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.965695 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.965869 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.965971 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.966155 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.966249 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.966441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.966526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.966720 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.966804 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.967022 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.967109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.967315 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.967398 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.983221 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.983434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.986259 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.986357 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.989180 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.989299 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.990894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.990995 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.992408 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.992541 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:11.992699 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.992926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.993020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.993210 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.999524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:11.999706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.001003 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:12.001283 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:12.003905 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.004165 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.004270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.004456 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.004542 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.004742 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.004826 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.005018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.012711 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:12.013004 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.013140 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:12.013289 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.013432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.013714 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.020227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81868) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:12.020470 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81869) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:04:12.020558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.020754 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.020848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.021038 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.021123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.021316 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.021399 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.021589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.021674 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.021861 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.021947 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.022155 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.022250 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.022441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.022532 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.022717 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.022926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.023015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.023211 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.023298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.023487 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.023571 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.024269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.024408 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81868) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.024535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.024813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81869) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:04:12.059283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560947abe9c0 pid: 81868 (default app) Aug 31 19:04:12.059416 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560947abe9c0 pid: 81869 (default app) Aug 31 19:04:12.062708 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a78c97-f0e7-41ce-b274-e12e3bbb697c None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:12.066168 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a78c97-f0e7-41ce-b274-e12e3bbb697c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:12.066168 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a78c97-f0e7-41ce-b274-e12e3bbb697c None None] PATH_INFO: `/v3` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:12.067456 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1/1] 10.222.0.59 () {50 vars in 791 bytes} [Wed Aug 31 19:04:12 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:19.502779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef15e7-ad65-4996-b34d-f16a0b294906 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:19.503356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef15e7-ad65-4996-b34d-f16a0b294906 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:19.503517 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef15e7-ad65-4996-b34d-f16a0b294906 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:19.508139 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1/2] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:04:19.513625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928a9ca-f9b7-4eda-8644-131d36dcf5a4 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:19.514119 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928a9ca-f9b7-4eda-8644-131d36dcf5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:19.514218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928a9ca-f9b7-4eda-8644-131d36dcf5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:19.584763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: INFO dbcounter [None req-c928a9ca-f9b7-4eda-8644-131d36dcf5a4 None None] Registered counter for database keystone Aug 31 19:04:19.604933 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG oslo_db.sqlalchemy.engines [None req-c928a9ca-f9b7-4eda-8644-131d36dcf5a4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81868) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 19:04:19.611259 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writer thread running {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 19:04:19.794250 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG passlib.handlers.bcrypt [None req-c928a9ca-f9b7-4eda-8644-131d36dcf5a4 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81868) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 19:04:19.794637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG passlib.handlers.bcrypt [None req-c928a9ca-f9b7-4eda-8644-131d36dcf5a4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81868) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 19:04:19.853863 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-c928a9ca-f9b7-4eda-8644-131d36dcf5a4 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:19.855449 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c928a9ca-f9b7-4eda-8644-131d36dcf5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:19.985284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 2/3] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 31 19:04:20.458408 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef15e7-ad65-4996-b34d-f16a0b294906 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:20.529465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: INFO dbcounter [None req-c5ef15e7-ad65-4996-b34d-f16a0b294906 None None] Registered counter for database keystone Aug 31 19:04:20.551318 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG oslo_db.sqlalchemy.engines [None req-c5ef15e7-ad65-4996-b34d-f16a0b294906 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81869) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 19:04:20.558344 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writer thread running {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 19:04:20.679334 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef15e7-ad65-4996-b34d-f16a0b294906 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:20.684557 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ecb52-55f9-465b-94db-629b4b5b19e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:20.685212 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ecb52-55f9-465b-94db-629b4b5b19e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:20.685491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ecb52-55f9-465b-94db-629b4b5b19e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:20.685763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ecb52-55f9-465b-94db-629b4b5b19e3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:20.694071 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ecb52-55f9-465b-94db-629b4b5b19e3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:20.746703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fd4ecb52-55f9-465b-94db-629b4b5b19e3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 19:04:20.747091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ecb52-55f9-465b-94db-629b4b5b19e3 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:20.756982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 2/4] 10.222.0.59 () {58 vars in 1324 bytes} [Wed Aug 31 19:04:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 19:04:20.762988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928a9ca-f9b7-4eda-8644-131d36dcf5a4 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:20.777447 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928a9ca-f9b7-4eda-8644-131d36dcf5a4 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:20.781889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c9d75-5a6d-4ec9-adcf-3ec4ad92b88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:20.782956 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c9d75-5a6d-4ec9-adcf-3ec4ad92b88e admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:20.783238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c9d75-5a6d-4ec9-adcf-3ec4ad92b88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:20.783509 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c9d75-5a6d-4ec9-adcf-3ec4ad92b88e admin admin] PATH_INFO: `/v3/projects/c9e5945a1a4d41fca11373bc396c02fd` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:20.785061 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c9d75-5a6d-4ec9-adcf-3ec4ad92b88e admin admin] RBAC: Authorizing `identity:get_project(project_id=c9e5945a1a4d41fca11373bc396c02fd)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:20.838436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a47c9d75-5a6d-4ec9-adcf-3ec4ad92b88e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 19:04:20.838774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c9d75-5a6d-4ec9-adcf-3ec4ad92b88e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:20.843372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 3/5] 10.222.0.59 () {56 vars in 1207 bytes} [Wed Aug 31 19:04:20 2022] GET /identity/v3/projects/c9e5945a1a4d41fca11373bc396c02fd => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:22.743904 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a72966-5eab-4f1b-b436-27b17c5fe651 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.744427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a72966-5eab-4f1b-b436-27b17c5fe651 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.744427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a72966-5eab-4f1b-b436-27b17c5fe651 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.748535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 3/6] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:22.753882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d9311-192b-4302-94fa-6cad942dccf3 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.754284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d9311-192b-4302-94fa-6cad942dccf3 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.754438 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d9311-192b-4302-94fa-6cad942dccf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.775993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-322d9311-192b-4302-94fa-6cad942dccf3 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:22.777536 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-322d9311-192b-4302-94fa-6cad942dccf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:22.785632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 4/7] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 19:04:23.058431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a72966-5eab-4f1b-b436-27b17c5fe651 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:23.071131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a72966-5eab-4f1b-b436-27b17c5fe651 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:23.075401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f578f2a-6229-4ebb-aafc-942357eda97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:23.076012 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f578f2a-6229-4ebb-aafc-942357eda97b admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:23.076297 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f578f2a-6229-4ebb-aafc-942357eda97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:23.076574 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f578f2a-6229-4ebb-aafc-942357eda97b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:23.084285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f578f2a-6229-4ebb-aafc-942357eda97b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:23.086187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f578f2a-6229-4ebb-aafc-942357eda97b admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:23.095292 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 4/8] 10.222.0.59 () {58 vars in 1324 bytes} [Wed Aug 31 19:04:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:04:23.100980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d9311-192b-4302-94fa-6cad942dccf3 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:23.110156 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d9311-192b-4302-94fa-6cad942dccf3 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:23.114430 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb141dc4-ff1a-4421-9b57-1a8f69f187e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:23.115345 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb141dc4-ff1a-4421-9b57-1a8f69f187e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:23.115623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb141dc4-ff1a-4421-9b57-1a8f69f187e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:23.115894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb141dc4-ff1a-4421-9b57-1a8f69f187e1 admin admin] PATH_INFO: `/v3/users/30d09a6b4f0047b4ba7decc0e37bf0cf` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:23.117737 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb141dc4-ff1a-4421-9b57-1a8f69f187e1 admin admin] RBAC: Authorizing `identity:get_user(user_id=30d09a6b4f0047b4ba7decc0e37bf0cf)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:23.119800 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb141dc4-ff1a-4421-9b57-1a8f69f187e1 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:23.121096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 5/9] 10.222.0.59 () {56 vars in 1198 bytes} [Wed Aug 31 19:04:23 2022] GET /identity/v3/users/30d09a6b4f0047b4ba7decc0e37bf0cf => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:25.075662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345b182-3e9f-4d73-a88c-e6d0a3e5fd8e None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:25.076190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345b182-3e9f-4d73-a88c-e6d0a3e5fd8e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:25.076190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345b182-3e9f-4d73-a88c-e6d0a3e5fd8e None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:25.077506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 5/10] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:25.082530 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce00dc-4d4a-4089-b8df-56dcde8baea4 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:25.082974 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce00dc-4d4a-4089-b8df-56dcde8baea4 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:25.083248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce00dc-4d4a-4089-b8df-56dcde8baea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:25.102188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-57ce00dc-4d4a-4089-b8df-56dcde8baea4 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:25.103402 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-57ce00dc-4d4a-4089-b8df-56dcde8baea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:25.174158 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01770307-1e97-49d6-84c0-be1093fac97d None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:25.174448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01770307-1e97-49d6-84c0-be1093fac97d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:25.174725 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01770307-1e97-49d6-84c0-be1093fac97d None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:25.176131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 6/11] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:25.181515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae069126-36c4-4b44-b971-1d044651690d None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:25.181794 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae069126-36c4-4b44-b971-1d044651690d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:25.182076 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae069126-36c4-4b44-b971-1d044651690d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:25.184677 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 6/12] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:25.225726 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG passlib.handlers.bcrypt [None req-ae069126-36c4-4b44-b971-1d044651690d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81869) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 19:04:25.226141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG passlib.handlers.bcrypt [None req-ae069126-36c4-4b44-b971-1d044651690d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81869) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 19:04:25.276225 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ae069126-36c4-4b44-b971-1d044651690d None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:25.277821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ae069126-36c4-4b44-b971-1d044651690d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:25.380537 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 7/13] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:25.450080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce00dc-4d4a-4089-b8df-56dcde8baea4 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:25.474179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:25.558224 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce00dc-4d4a-4089-b8df-56dcde8baea4 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:25.592931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abd4e1-2581-42ea-8d14-973420960223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:25.593578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abd4e1-2581-42ea-8d14-973420960223 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:25.593864 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abd4e1-2581-42ea-8d14-973420960223 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:25.594147 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abd4e1-2581-42ea-8d14-973420960223 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:25.606501 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8abd4e1-2581-42ea-8d14-973420960223 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:25.615463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-d8abd4e1-2581-42ea-8d14-973420960223 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:25.615463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:25.615463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:25.615463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:25.615463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:25.615463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:25.615463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:25.615463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:25.615463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:25.615463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:25.615463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:25.615463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:25.615463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:25.615463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:25.615463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:25.615463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:25.616487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:25.616487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:25.616487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:25.616487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:25.616487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:25.616487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:25.616487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:25.616487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:25.616487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:25.616487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:25.616487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:25.616487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:25.616487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:25.616487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:25.616487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:25.616487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:25.617439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:25.617439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:25.617439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:25.617439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:25.617439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:25.617439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:25.617439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:25.617439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:25.617439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:25.617439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:25.617439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:25.617439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:04:25.617439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 7/14] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:25.622410 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae069126-36c4-4b44-b971-1d044651690d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:25.651645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abd4e1-2581-42ea-8d14-973420960223 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:25.726095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae069126-36c4-4b44-b971-1d044651690d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:25.728681 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:25.735213 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:25.753526 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abd4e1-2581-42ea-8d14-973420960223 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:25.765647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e8a82-91d5-49d0-b85e-cc05b076c106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:25.766520 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e8a82-91d5-49d0-b85e-cc05b076c106 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:25.766800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e8a82-91d5-49d0-b85e-cc05b076c106 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:25.767105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e8a82-91d5-49d0-b85e-cc05b076c106 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:25.768085 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e8a82-91d5-49d0-b85e-cc05b076c106 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:25.768388 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e8a82-91d5-49d0-b85e-cc05b076c106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:25.769913 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e8a82-91d5-49d0-b85e-cc05b076c106 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:25.782987 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 8/15] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:25.788445 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e8a82-91d5-49d0-b85e-cc05b076c106 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:25.789962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd6117-5d19-4803-8e80-a89dc9cc3988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:25.790681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd6117-5d19-4803-8e80-a89dc9cc3988 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:25.790996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd6117-5d19-4803-8e80-a89dc9cc3988 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:25.791271 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd6117-5d19-4803-8e80-a89dc9cc3988 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:25.863950 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd6117-5d19-4803-8e80-a89dc9cc3988 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:25.865694 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd6117-5d19-4803-8e80-a89dc9cc3988 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:25.896594 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e8a82-91d5-49d0-b85e-cc05b076c106 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:25.932635 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6faf69-0e2c-4e76-97ed-22ca21351d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:25.933427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6faf69-0e2c-4e76-97ed-22ca21351d34 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:25.933707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6faf69-0e2c-4e76-97ed-22ca21351d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:25.933981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6faf69-0e2c-4e76-97ed-22ca21351d34 None admin] PATH_INFO: `/v3/users/30d09a6b4f0047b4ba7decc0e37bf0cf` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:25.935568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6faf69-0e2c-4e76-97ed-22ca21351d34 None admin] RBAC: Authorizing `identity:get_user(user_id=30d09a6b4f0047b4ba7decc0e37bf0cf)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:25.937024 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6faf69-0e2c-4e76-97ed-22ca21351d34 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:25.938640 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 9/16] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:25 2022] GET /identity/v3/users/30d09a6b4f0047b4ba7decc0e37bf0cf => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:25.944138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6faf69-0e2c-4e76-97ed-22ca21351d34 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:25.947556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:25.965787 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 8/17] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 19:04:25.971483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd6117-5d19-4803-8e80-a89dc9cc3988 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:25.979612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:26.047552 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6faf69-0e2c-4e76-97ed-22ca21351d34 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:26.073990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd6117-5d19-4803-8e80-a89dc9cc3988 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:26.083877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477e434-616e-433c-aec2-ec4a5887c3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:26.084515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477e434-616e-433c-aec2-ec4a5887c3eb None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:26.084800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477e434-616e-433c-aec2-ec4a5887c3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:26.085077 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477e434-616e-433c-aec2-ec4a5887c3eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:26.086741 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7477e434-616e-433c-aec2-ec4a5887c3eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:26.088223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7477e434-616e-433c-aec2-ec4a5887c3eb None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:26.089782 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 10/18] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:26.095518 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477e434-616e-433c-aec2-ec4a5887c3eb None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:26.109928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c8c46-7287-4a83-96e1-845497a83d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:26.110565 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c8c46-7287-4a83-96e1-845497a83d12 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:26.110840 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c8c46-7287-4a83-96e1-845497a83d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:26.111147 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c8c46-7287-4a83-96e1-845497a83d12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:26.121590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c8c46-7287-4a83-96e1-845497a83d12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:26.123094 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c8c46-7287-4a83-96e1-845497a83d12 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:26.139220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-0a8c8c46-7287-4a83-96e1-845497a83d12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:26.139220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:26.139220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:26.139220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:26.139220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:26.139220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:26.139220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:26.139220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:26.139220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:26.139220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:26.139220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:26.139220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:26.139220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:26.139220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:26.139220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:26.139220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:26.140258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:26.140258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:26.140258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:26.140258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:26.140258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:26.140258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:26.140258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:26.140258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:26.140258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:26.140258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:26.140258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:26.140258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:26.140258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:26.140258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:26.140258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:26.140258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:26.140258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:26.141266 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:26.141266 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:26.141266 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:04:26.141266 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 9/19] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:26.146084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c8c46-7287-4a83-96e1-845497a83d12 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:26.187985 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:26.200356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477e434-616e-433c-aec2-ec4a5887c3eb None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:26.202347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:26.233789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3657641-595f-431b-ae84-918141f885be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:26.234406 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3657641-595f-431b-ae84-918141f885be None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:26.234690 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3657641-595f-431b-ae84-918141f885be None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:26.234992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3657641-595f-431b-ae84-918141f885be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:26.235787 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3657641-595f-431b-ae84-918141f885be None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.domain.id=default, user.id=30d09a6b4f0047b4ba7decc0e37bf0cf) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:26.236027 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3657641-595f-431b-ae84-918141f885be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:26.237483 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3657641-595f-431b-ae84-918141f885be None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:26.246587 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 11/20] 10.222.0.59 () {56 vars in 1322 bytes} [Wed Aug 31 19:04:26 2022] GET /identity/v3/role_assignments?user.id=30d09a6b4f0047b4ba7decc0e37bf0cf&scope.domain.id=default&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:26.252780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c8c46-7287-4a83-96e1-845497a83d12 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:26.296734 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219d6ae-bc33-4c82-b538-81a284db4c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:26.297380 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219d6ae-bc33-4c82-b538-81a284db4c01 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:26.297664 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219d6ae-bc33-4c82-b538-81a284db4c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:26.297943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219d6ae-bc33-4c82-b538-81a284db4c01 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:26.298623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219d6ae-bc33-4c82-b538-81a284db4c01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:26.298973 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219d6ae-bc33-4c82-b538-81a284db4c01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:26.300425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219d6ae-bc33-4c82-b538-81a284db4c01 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:26.319862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 10/21] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:28.443805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc2e8e-e4c0-4d6b-9cb8-b1f876575ea8 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:28.444339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc2e8e-e4c0-4d6b-9cb8-b1f876575ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:28.444339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc2e8e-e4c0-4d6b-9cb8-b1f876575ea8 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:28.445845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 12/22] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:28.451258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ca18b-d833-4c62-9257-92eaf9d117c7 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:28.451699 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ca18b-d833-4c62-9257-92eaf9d117c7 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:28.451896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ca18b-d833-4c62-9257-92eaf9d117c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:28.474996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a44ca18b-d833-4c62-9257-92eaf9d117c7 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:28.476820 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a44ca18b-d833-4c62-9257-92eaf9d117c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:28.566178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 11/23] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:28.583678 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6263b-dc3b-44cc-9722-3bbb9aa7cdfc None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:28.584310 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6263b-dc3b-44cc-9722-3bbb9aa7cdfc None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:28.584796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6263b-dc3b-44cc-9722-3bbb9aa7cdfc None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:28.587445 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 13/24] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:28.592988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f36af-1e65-4ff2-92a5-0841b3ef590c None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:28.593557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f36af-1e65-4ff2-92a5-0841b3ef590c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:28.594051 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f36af-1e65-4ff2-92a5-0841b3ef590c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:28.613589 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-1b0f36af-1e65-4ff2-92a5-0841b3ef590c None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:28.615424 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-1b0f36af-1e65-4ff2-92a5-0841b3ef590c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:28.638899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:28.707592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 12/25] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:28.948761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74ad4c-f88b-4084-804e-3e60675b2a20 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:28.949471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74ad4c-f88b-4084-804e-3e60675b2a20 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:28.949960 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74ad4c-f88b-4084-804e-3e60675b2a20 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:28.951821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 14/26] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:28.957018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ff2ab-d5f4-4f3a-afb3-81d4875f7967 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:28.957318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ff2ab-d5f4-4f3a-afb3-81d4875f7967 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:28.957603 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ff2ab-d5f4-4f3a-afb3-81d4875f7967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:28.984829 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-e05ff2ab-d5f4-4f3a-afb3-81d4875f7967 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:28.986019 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e05ff2ab-d5f4-4f3a-afb3-81d4875f7967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:29.046702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74ad4c-f88b-4084-804e-3e60675b2a20 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:29.075444 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 13/27] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:29.197098 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23c33e-a2b9-44c7-bece-01f7d5a718ff None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.197742 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23c33e-a2b9-44c7-bece-01f7d5a718ff None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.198229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23c33e-a2b9-44c7-bece-01f7d5a718ff None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.204043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 14/28] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:04:29.209923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c0477-fb31-4ba9-b934-51484ac7e797 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.210736 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c0477-fb31-4ba9-b934-51484ac7e797 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.211527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c0477-fb31-4ba9-b934-51484ac7e797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.218306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74ad4c-f88b-4084-804e-3e60675b2a20 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:29.239123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-4d4c0477-fb31-4ba9-b934-51484ac7e797 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:29.246778 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4d4c0477-fb31-4ba9-b934-51484ac7e797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:29.314230 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90c121-79b5-4e7c-ac04-252c67d2b98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:29.314902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90c121-79b5-4e7c-ac04-252c67d2b98e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.315191 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90c121-79b5-4e7c-ac04-252c67d2b98e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.315463 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90c121-79b5-4e7c-ac04-252c67d2b98e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.340042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 15/29] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:29.345038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c0477-fb31-4ba9-b934-51484ac7e797 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:29.349216 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e90c121-79b5-4e7c-ac04-252c67d2b98e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:29.366985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-8e90c121-79b5-4e7c-ac04-252c67d2b98e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:29.366985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:29.366985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:29.366985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:29.366985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:29.366985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:29.366985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:29.366985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:29.366985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:29.366985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:29.366985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:29.366985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:29.366985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:29.366985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:29.366985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:29.366985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:29.368104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:29.368104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:29.368104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:29.368104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:29.368104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:29.368104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:29.368104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:29.368104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:29.368104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:29.368104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:29.368104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:29.368104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:29.368104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:29.368104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:29.368104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:29.368104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:29.369143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:29.369143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:29.369143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:29.369143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:29.369143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:29.369143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:29.369143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:29.369143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:29.369143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:29.369143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:29.369143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:29.369143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:04:29.369899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:29.371402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 15/30] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:29.377403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90c121-79b5-4e7c-ac04-252c67d2b98e None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:29.432757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:29.462988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c0477-fb31-4ba9-b934-51484ac7e797 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:29.506332 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8df62-4aad-482b-acfb-4dcfa149ba63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:29.507783 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8df62-4aad-482b-acfb-4dcfa149ba63 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.508392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8df62-4aad-482b-acfb-4dcfa149ba63 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.508869 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8df62-4aad-482b-acfb-4dcfa149ba63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.521767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8df62-4aad-482b-acfb-4dcfa149ba63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:29.524044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8df62-4aad-482b-acfb-4dcfa149ba63 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:29.542171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-c5d8df62-4aad-482b-acfb-4dcfa149ba63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:29.542171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:29.542171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:29.542171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:29.542171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:29.542171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:29.542171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:29.542171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:29.542171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:29.542171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:29.542171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:29.542171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:29.542171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:29.542171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:29.542171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:29.542171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:29.543152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:29.543152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:29.543152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:29.543152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:29.543152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:29.543152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:29.543152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:29.543152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:29.543152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:29.543152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:29.543152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:29.543152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:29.543152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:29.543152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:29.543152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:29.543152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:29.543152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:29.543962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:29.543962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:29.543962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:04:29.546613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 16/31] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:29.552440 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8df62-4aad-482b-acfb-4dcfa149ba63 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:29.606580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90c121-79b5-4e7c-ac04-252c67d2b98e None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:29.644329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:29.685886 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8df62-4aad-482b-acfb-4dcfa149ba63 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:29.688958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dac1e-af77-49a3-9896-70e37022530a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:29.690047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dac1e-af77-49a3-9896-70e37022530a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.690047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dac1e-af77-49a3-9896-70e37022530a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.690221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dac1e-af77-49a3-9896-70e37022530a None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.690910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01dac1e-af77-49a3-9896-70e37022530a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:29.691227 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01dac1e-af77-49a3-9896-70e37022530a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:29.692622 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01dac1e-af77-49a3-9896-70e37022530a None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:29.715201 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 16/32] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:29.715722 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dac1e-af77-49a3-9896-70e37022530a None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:29.726721 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac5199-b664-45cd-aa5f-9c642c1b2526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:29.727731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac5199-b664-45cd-aa5f-9c642c1b2526 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.729032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac5199-b664-45cd-aa5f-9c642c1b2526 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.729512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac5199-b664-45cd-aa5f-9c642c1b2526 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.730376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac5199-b664-45cd-aa5f-9c642c1b2526 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:29.730911 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac5199-b664-45cd-aa5f-9c642c1b2526 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:29.732507 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac5199-b664-45cd-aa5f-9c642c1b2526 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:29.748148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 17/33] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:29.749615 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac5199-b664-45cd-aa5f-9c642c1b2526 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:29.870479 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac5199-b664-45cd-aa5f-9c642c1b2526 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:29.895696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:29.905302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:29.909895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dac1e-af77-49a3-9896-70e37022530a None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:29.917127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3873f-f5c2-4575-aac9-ee3b716fad23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:29.918698 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3873f-f5c2-4575-aac9-ee3b716fad23 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.919202 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3873f-f5c2-4575-aac9-ee3b716fad23 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.919680 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3873f-f5c2-4575-aac9-ee3b716fad23 None admin] PATH_INFO: `/v3/users/30d09a6b4f0047b4ba7decc0e37bf0cf` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.921477 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3873f-f5c2-4575-aac9-ee3b716fad23 None admin] RBAC: Authorizing `identity:get_user(user_id=30d09a6b4f0047b4ba7decc0e37bf0cf)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:29.923839 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3873f-f5c2-4575-aac9-ee3b716fad23 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:29.926863 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 18/34] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:29 2022] GET /identity/v3/users/30d09a6b4f0047b4ba7decc0e37bf0cf => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:29.927907 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3873f-f5c2-4575-aac9-ee3b716fad23 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:29.964357 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98bbed-2005-45f8-8fc5-15418339ca7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:29.965308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98bbed-2005-45f8-8fc5-15418339ca7c None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.965535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98bbed-2005-45f8-8fc5-15418339ca7c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.965809 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98bbed-2005-45f8-8fc5-15418339ca7c None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.966492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98bbed-2005-45f8-8fc5-15418339ca7c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:29.968659 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98bbed-2005-45f8-8fc5-15418339ca7c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:30.041269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3873f-f5c2-4575-aac9-ee3b716fad23 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:30.077660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04028cde-95f4-45ac-809b-2ebf7759223d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:30.078284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04028cde-95f4-45ac-809b-2ebf7759223d None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.078572 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04028cde-95f4-45ac-809b-2ebf7759223d None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.078858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04028cde-95f4-45ac-809b-2ebf7759223d None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.079549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04028cde-95f4-45ac-809b-2ebf7759223d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:30.080998 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04028cde-95f4-45ac-809b-2ebf7759223d None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:30.178905 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 17/35] 10.222.0.59 () {60 vars in 1131 bytes} [Wed Aug 31 19:04:29 2022] POST /identity/v3/roles => generated 225 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:04:30.178905 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98bbed-2005-45f8-8fc5-15418339ca7c None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:30.307414 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98bbed-2005-45f8-8fc5-15418339ca7c None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:30.353041 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c01a0-f008-4078-8546-5fef5abb3511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:30.353899 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c01a0-f008-4078-8546-5fef5abb3511 None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.354288 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c01a0-f008-4078-8546-5fef5abb3511 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.354660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c01a0-f008-4078-8546-5fef5abb3511 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.355542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c01a0-f008-4078-8546-5fef5abb3511 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:30.357383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c01a0-f008-4078-8546-5fef5abb3511 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:30.413167 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 19/36] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 31 19:04:29 2022] POST /identity/v3/projects => generated 301 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:04:30.414618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04028cde-95f4-45ac-809b-2ebf7759223d None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:30.438493 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 18/37] 10.222.0.59 () {60 vars in 1131 bytes} [Wed Aug 31 19:04:30 2022] POST /identity/v3/roles => generated 219 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:04:30.439659 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9674e-f1b6-4e21-b67a-98044cbfbe29 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.439951 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9674e-f1b6-4e21-b67a-98044cbfbe29 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.440222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9674e-f1b6-4e21-b67a-98044cbfbe29 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.441636 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 19/38] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:30.448204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced225e5-ec6f-4e6a-820d-16ddda9ba749 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.448511 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced225e5-ec6f-4e6a-820d-16ddda9ba749 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.448752 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced225e5-ec6f-4e6a-820d-16ddda9ba749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.487809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:30.490258 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ced225e5-ec6f-4e6a-820d-16ddda9ba749 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:30.492002 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ced225e5-ec6f-4e6a-820d-16ddda9ba749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:30.494640 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:30.560567 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04028cde-95f4-45ac-809b-2ebf7759223d None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:30.580433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 20/39] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:30.582138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c62a80-b70a-4286-843b-1724ad57d98e None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.582624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c62a80-b70a-4286-843b-1724ad57d98e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.583115 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c62a80-b70a-4286-843b-1724ad57d98e None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.584716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 21/40] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:30.594937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb01999-eb51-41ef-8b0b-f7861be57d29 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.594937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb01999-eb51-41ef-8b0b-f7861be57d29 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.594937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb01999-eb51-41ef-8b0b-f7861be57d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.622789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9edea-1a97-4ff9-bc4b-6d76888e316f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:30.623513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9edea-1a97-4ff9-bc4b-6d76888e316f None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.623802 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9edea-1a97-4ff9-bc4b-6d76888e316f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.624085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9edea-1a97-4ff9-bc4b-6d76888e316f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.627897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9edea-1a97-4ff9-bc4b-6d76888e316f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:30.629787 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9edea-1a97-4ff9-bc4b-6d76888e316f None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:30.631209 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 20/41] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:30.632856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-bbb01999-eb51-41ef-8b0b-f7861be57d29 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:30.634063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-bbb01999-eb51-41ef-8b0b-f7861be57d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:30.637224 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9edea-1a97-4ff9-bc4b-6d76888e316f None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:30.719658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 22/42] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:30.776798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9edea-1a97-4ff9-bc4b-6d76888e316f None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:30.792572 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:30.816525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae816b5-00d1-472b-915a-821707a5c1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:30.817126 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae816b5-00d1-472b-915a-821707a5c1ec None admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.817409 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae816b5-00d1-472b-915a-821707a5c1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.817685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae816b5-00d1-472b-915a-821707a5c1ec None admin] PATH_INFO: `/v3/domains/default/users/30d09a6b4f0047b4ba7decc0e37bf0cf/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.820465 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae816b5-00d1-472b-915a-821707a5c1ec None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=30d09a6b4f0047b4ba7decc0e37bf0cf, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:30.821921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae816b5-00d1-472b-915a-821707a5c1ec None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:30.863841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb01999-eb51-41ef-8b0b-f7861be57d29 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:30.930162 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:30.948594 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 21/43] 10.222.0.59 () {58 vars in 1361 bytes} [Wed Aug 31 19:04:30 2022] PUT /identity/v3/domains/default/users/30d09a6b4f0047b4ba7decc0e37bf0cf/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:30.976570 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb01999-eb51-41ef-8b0b-f7861be57d29 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:31.004366 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae816b5-00d1-472b-915a-821707a5c1ec None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:31.017490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d9925-e023-4491-91eb-8b767920c7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:31.018131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d9925-e023-4491-91eb-8b767920c7b2 None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:31.018418 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d9925-e023-4491-91eb-8b767920c7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:31.018693 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d9925-e023-4491-91eb-8b767920c7b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:31.019426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d9925-e023-4491-91eb-8b767920c7b2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:31.020842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d9925-e023-4491-91eb-8b767920c7b2 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:31.096978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 23/44] 10.222.0.59 () {60 vars in 1131 bytes} [Wed Aug 31 19:04:30 2022] POST /identity/v3/roles => generated 223 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:04:31.120672 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae816b5-00d1-472b-915a-821707a5c1ec None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:31.161382 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec07107-2df7-4184-973e-56941dbd91d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:31.162102 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec07107-2df7-4184-973e-56941dbd91d5 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:31.162391 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec07107-2df7-4184-973e-56941dbd91d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:31.162667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec07107-2df7-4184-973e-56941dbd91d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:31.163939 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec07107-2df7-4184-973e-56941dbd91d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:31.165505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec07107-2df7-4184-973e-56941dbd91d5 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:31.167472 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 22/45] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:31.173059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d9925-e023-4491-91eb-8b767920c7b2 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:31.296678 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d9925-e023-4491-91eb-8b767920c7b2 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:31.326038 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:31.333922 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c191069-9be8-45e5-90c3-c23dede277f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:31.334546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c191069-9be8-45e5-90c3-c23dede277f8 None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:31.334833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c191069-9be8-45e5-90c3-c23dede277f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:31.335181 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c191069-9be8-45e5-90c3-c23dede277f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:31.335924 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c191069-9be8-45e5-90c3-c23dede277f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:31.337381 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c191069-9be8-45e5-90c3-c23dede277f8 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:31.475453 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 24/46] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 31 19:04:31 2022] POST /identity/v3/projects => generated 312 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:04:32.897731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c514b-2ec4-4350-ab9c-8ebafe6ac9d6 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:32.898292 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c514b-2ec4-4350-ab9c-8ebafe6ac9d6 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:32.898486 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c514b-2ec4-4350-ab9c-8ebafe6ac9d6 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:32.899963 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 23/47] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:32.905292 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f00cf-36dd-448f-a58e-7e2a9cd38ea7 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:32.905706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f00cf-36dd-448f-a58e-7e2a9cd38ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:32.905843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f00cf-36dd-448f-a58e-7e2a9cd38ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:32.924891 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-202f00cf-36dd-448f-a58e-7e2a9cd38ea7 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:32.926081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-202f00cf-36dd-448f-a58e-7e2a9cd38ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:32.993447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 25/48] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:33.267792 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c514b-2ec4-4350-ab9c-8ebafe6ac9d6 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:33.314738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:33.373220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c514b-2ec4-4350-ab9c-8ebafe6ac9d6 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:33.400675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97fa09-0344-46be-acc9-6bf03c8b64bd None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:33.400968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97fa09-0344-46be-acc9-6bf03c8b64bd None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:33.401248 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97fa09-0344-46be-acc9-6bf03c8b64bd None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:33.402729 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 26/49] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:04:33.407549 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d4544-60b7-40b5-9f7b-b4befe827a81 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:33.407836 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d4544-60b7-40b5-9f7b-b4befe827a81 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:33.408109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d4544-60b7-40b5-9f7b-b4befe827a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:33.408663 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de72f45-ff4b-4d54-b086-2c439eaf145a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:33.409310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de72f45-ff4b-4d54-b086-2c439eaf145a None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:33.409602 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de72f45-ff4b-4d54-b086-2c439eaf145a None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:33.409881 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de72f45-ff4b-4d54-b086-2c439eaf145a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:33.421495 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de72f45-ff4b-4d54-b086-2c439eaf145a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:33.426425 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-c39d4544-60b7-40b5-9f7b-b4befe827a81 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:33.426555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-9de72f45-ff4b-4d54-b086-2c439eaf145a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:33.426555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:33.426555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:33.426555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:33.426555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:33.426555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:33.426555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:33.426555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:33.426555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:33.426555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:33.426555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:33.426555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:33.426555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:33.426555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:33.426555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:33.426555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:33.427566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:33.427566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:33.427566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:33.427566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:33.427566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:33.427566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:33.427566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:33.427566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:33.427566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:33.427566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:33.427566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:33.427566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:33.427566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:33.427566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:33.427566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:33.427566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:33.428351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:33.428351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:33.428351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:33.428351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:33.428351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:33.428351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:33.428351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:33.428351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:33.428351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:33.428351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:33.428351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:33.428351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:04:33.428351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 24/50] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:33.429606 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c39d4544-60b7-40b5-9f7b-b4befe827a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:33.434189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de72f45-ff4b-4d54-b086-2c439eaf145a None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:33.497240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 27/51] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:33.534723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de72f45-ff4b-4d54-b086-2c439eaf145a None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:33.536185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:33.570107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6756f-a6b9-467e-a245-52e971860d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:33.570730 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6756f-a6b9-467e-a245-52e971860d0a None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:33.571048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6756f-a6b9-467e-a245-52e971860d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:33.571331 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6756f-a6b9-467e-a245-52e971860d0a None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:33.572008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6756f-a6b9-467e-a245-52e971860d0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:33.572320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6756f-a6b9-467e-a245-52e971860d0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:33.573726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6756f-a6b9-467e-a245-52e971860d0a None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:33.586283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 25/52] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:33.592058 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d4544-60b7-40b5-9f7b-b4befe827a81 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:33.633915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:33.691343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d4544-60b7-40b5-9f7b-b4befe827a81 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:33.725533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1254fd-2de2-419e-836a-a7b18078a6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:33.726832 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1254fd-2de2-419e-836a-a7b18078a6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:33.727156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1254fd-2de2-419e-836a-a7b18078a6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:33.727434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1254fd-2de2-419e-836a-a7b18078a6a6 None admin] PATH_INFO: `/v3/users/30d09a6b4f0047b4ba7decc0e37bf0cf` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:33.728571 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1254fd-2de2-419e-836a-a7b18078a6a6 None admin] RBAC: Authorizing `identity:get_user(user_id=30d09a6b4f0047b4ba7decc0e37bf0cf)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:33.730218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1254fd-2de2-419e-836a-a7b18078a6a6 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:33.731830 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 28/53] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:33 2022] GET /identity/v3/users/30d09a6b4f0047b4ba7decc0e37bf0cf => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:33.737470 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6756f-a6b9-467e-a245-52e971860d0a None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:33.765190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1254fd-2de2-419e-836a-a7b18078a6a6 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:33.840879 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6756f-a6b9-467e-a245-52e971860d0a None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:33.868747 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1254fd-2de2-419e-836a-a7b18078a6a6 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:33.875165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811488a-3856-4451-b4fa-ebaa7bc1e881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:33.875818 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811488a-3856-4451-b4fa-ebaa7bc1e881 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:33.876110 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811488a-3856-4451-b4fa-ebaa7bc1e881 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:33.876391 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811488a-3856-4451-b4fa-ebaa7bc1e881 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:33.877627 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811488a-3856-4451-b4fa-ebaa7bc1e881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:33.878273 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:33.879153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811488a-3856-4451-b4fa-ebaa7bc1e881 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:33.880797 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 26/54] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:33.887200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811488a-3856-4451-b4fa-ebaa7bc1e881 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:33.905632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290528b-4758-45b8-b7bb-89ba51948e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:33.906279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290528b-4758-45b8-b7bb-89ba51948e50 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:33.906568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290528b-4758-45b8-b7bb-89ba51948e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:33.906846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290528b-4758-45b8-b7bb-89ba51948e50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:33.908020 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290528b-4758-45b8-b7bb-89ba51948e50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:33.909626 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290528b-4758-45b8-b7bb-89ba51948e50 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:33.911235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 29/55] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:33.917224 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290528b-4758-45b8-b7bb-89ba51948e50 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:33.922726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:33.994388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811488a-3856-4451-b4fa-ebaa7bc1e881 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:34.020096 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290528b-4758-45b8-b7bb-89ba51948e50 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:34.029523 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d5858-198e-4f6f-9cdf-c4960a610d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:34.030138 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d5858-198e-4f6f-9cdf-c4960a610d00 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:34.030421 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d5858-198e-4f6f-9cdf-c4960a610d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:34.030692 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d5858-198e-4f6f-9cdf-c4960a610d00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:34.031450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d5858-198e-4f6f-9cdf-c4960a610d00 None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.domain.id=default, user.id=30d09a6b4f0047b4ba7decc0e37bf0cf) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:34.031760 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d5858-198e-4f6f-9cdf-c4960a610d00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:34.033200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d5858-198e-4f6f-9cdf-c4960a610d00 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:34.040535 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 27/56] 10.222.0.59 () {56 vars in 1322 bytes} [Wed Aug 31 19:04:33 2022] GET /identity/v3/role_assignments?user.id=30d09a6b4f0047b4ba7decc0e37bf0cf&scope.domain.id=default&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:34.056576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ba17f-6a98-4d9a-bfdb-053066527494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:34.057219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ba17f-6a98-4d9a-bfdb-053066527494 None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:34.057503 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ba17f-6a98-4d9a-bfdb-053066527494 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:34.057789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ba17f-6a98-4d9a-bfdb-053066527494 None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:34.058472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ba17f-6a98-4d9a-bfdb-053066527494 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:34.060022 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ba17f-6a98-4d9a-bfdb-053066527494 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:34.150116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 30/57] 10.222.0.59 () {60 vars in 1140 bytes} [Wed Aug 31 19:04:33 2022] POST /identity/v3/projects => generated 298 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:04:36.042211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f384e-4ebe-496d-8c6c-8e9c8afe2735 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:36.042760 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f384e-4ebe-496d-8c6c-8e9c8afe2735 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:36.042760 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f384e-4ebe-496d-8c6c-8e9c8afe2735 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:36.044187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 28/58] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:36.049178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f04978-378d-4aca-b12e-ab7e9ddef7ac None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:36.049578 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f04978-378d-4aca-b12e-ab7e9ddef7ac None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:36.049740 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f04978-378d-4aca-b12e-ab7e9ddef7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:36.068015 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-63f04978-378d-4aca-b12e-ab7e9ddef7ac None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:36.069246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-63f04978-378d-4aca-b12e-ab7e9ddef7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:36.092753 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:36.136520 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 31/59] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:36.408811 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f384e-4ebe-496d-8c6c-8e9c8afe2735 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:36.505114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:36.512317 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f384e-4ebe-496d-8c6c-8e9c8afe2735 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:36.547407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af212838-9c02-4372-aaa8-402f0a482486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:36.548047 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af212838-9c02-4372-aaa8-402f0a482486 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:36.548330 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af212838-9c02-4372-aaa8-402f0a482486 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:36.548605 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af212838-9c02-4372-aaa8-402f0a482486 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:36.549719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af212838-9c02-4372-aaa8-402f0a482486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:36.551212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af212838-9c02-4372-aaa8-402f0a482486 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:36.552786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 29/60] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:36.559090 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f04978-378d-4aca-b12e-ab7e9ddef7ac None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:36.660777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f04978-378d-4aca-b12e-ab7e9ddef7ac None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:36.801407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45689d8a-d5ff-4e42-8027-6d484e38404e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:36.802017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45689d8a-d5ff-4e42-8027-6d484e38404e None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:36.802304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45689d8a-d5ff-4e42-8027-6d484e38404e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:36.802581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45689d8a-d5ff-4e42-8027-6d484e38404e None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:36.803290 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45689d8a-d5ff-4e42-8027-6d484e38404e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:36.805381 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45689d8a-d5ff-4e42-8027-6d484e38404e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:36.906892 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 32/61] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:04:36 2022] POST /identity/v3/users => generated 275 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:04:39.914332 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586acd29-abb6-4773-950a-96f2124e787e None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:39.915204 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586acd29-abb6-4773-950a-96f2124e787e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:39.915622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586acd29-abb6-4773-950a-96f2124e787e None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:39.917155 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 30/62] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:39.922647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8042d9e-39ac-4285-8192-b6896a1bac01 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:39.923139 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8042d9e-39ac-4285-8192-b6896a1bac01 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:39.923421 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8042d9e-39ac-4285-8192-b6896a1bac01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:39.953691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-e8042d9e-39ac-4285-8192-b6896a1bac01 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:39.955816 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e8042d9e-39ac-4285-8192-b6896a1bac01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:39.978899 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:40.113967 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 33/63] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:40.316362 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1151a56-3b47-4224-9dd5-a0588608cf33 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.316989 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1151a56-3b47-4224-9dd5-a0588608cf33 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.317486 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1151a56-3b47-4224-9dd5-a0588608cf33 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.320022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 31/64] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:04:40.328646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc72289-ed4e-45c7-80cc-87c1fc24e76a None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.329135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc72289-ed4e-45c7-80cc-87c1fc24e76a None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.329610 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc72289-ed4e-45c7-80cc-87c1fc24e76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.367272 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60023b-1eca-4b0e-af2a-be227e78e5e5 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.367610 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60023b-1eca-4b0e-af2a-be227e78e5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.367894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60023b-1eca-4b0e-af2a-be227e78e5e5 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.369560 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 32/65] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:40.371483 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-8cc72289-ed4e-45c7-80cc-87c1fc24e76a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:40.373499 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8cc72289-ed4e-45c7-80cc-87c1fc24e76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:40.376435 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9390e7-6d54-4943-b771-6473a76c78f9 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.376905 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9390e7-6d54-4943-b771-6473a76c78f9 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.377199 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9390e7-6d54-4943-b771-6473a76c78f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.404762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-6f9390e7-6d54-4943-b771-6473a76c78f9 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:40.406330 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6f9390e7-6d54-4943-b771-6473a76c78f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:40.491205 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 34/66] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:40.492288 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4cbcc-2115-41bd-839a-04158701ee7c None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.492597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4cbcc-2115-41bd-839a-04158701ee7c None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.492861 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4cbcc-2115-41bd-839a-04158701ee7c None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.494796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 35/67] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:40.500221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc378d-d02b-4cc0-af3a-03599dddd007 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.500898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc378d-d02b-4cc0-af3a-03599dddd007 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.501367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc378d-d02b-4cc0-af3a-03599dddd007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.513848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 33/68] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:40.524308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-b9fc378d-d02b-4cc0-af3a-03599dddd007 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:40.525833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b9fc378d-d02b-4cc0-af3a-03599dddd007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:40.597793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 36/69] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:40.657421 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9390e7-6d54-4943-b771-6473a76c78f9 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:40.762285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:40.771167 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9390e7-6d54-4943-b771-6473a76c78f9 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:40.807630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac8157-e9ae-4aaf-9080-2dcf902d0cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:40.808622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac8157-e9ae-4aaf-9080-2dcf902d0cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.809161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac8157-e9ae-4aaf-9080-2dcf902d0cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.809638 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac8157-e9ae-4aaf-9080-2dcf902d0cd5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.821260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac8157-e9ae-4aaf-9080-2dcf902d0cd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:40.826473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-99ac8157-e9ae-4aaf-9080-2dcf902d0cd5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:40.826473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:40.826473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:40.826473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:40.826473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:40.826473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:40.826473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:40.826473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:40.826473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:40.826473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:40.826473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:40.826473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:40.826473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:40.826473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:40.826473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:40.826473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:40.827352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:40.827352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:40.827352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:40.827352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:40.827352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:40.827352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:40.827352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:40.827352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:40.827352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:40.827352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:40.827352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:40.827352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:40.827352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:40.827352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:40.827352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:40.827352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:40.828149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:40.828149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:40.828149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:40.828149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:40.828149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:40.828149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:40.828149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:40.828149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:40.828149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:40.828149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:40.828149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:40.828149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:04:40.830380 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 34/70] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:40.839979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc378d-d02b-4cc0-af3a-03599dddd007 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:40.843939 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:40.910071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac8157-e9ae-4aaf-9080-2dcf902d0cd5 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.004792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc378d-d02b-4cc0-af3a-03599dddd007 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.070238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853398e7-991b-4424-87d1-5659a49f2029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.071269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853398e7-991b-4424-87d1-5659a49f2029 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.071763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853398e7-991b-4424-87d1-5659a49f2029 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.072261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853398e7-991b-4424-87d1-5659a49f2029 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.073262 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853398e7-991b-4424-87d1-5659a49f2029 None admin] RBAC: Adding query filter params (name=member) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:41.073865 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853398e7-991b-4424-87d1-5659a49f2029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:41.075599 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853398e7-991b-4424-87d1-5659a49f2029 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:41.083574 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac8157-e9ae-4aaf-9080-2dcf902d0cd5 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.087238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 37/71] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:04:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:41.092915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853398e7-991b-4424-87d1-5659a49f2029 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.121968 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517923e3-b818-4ec0-ae15-360d0a98de60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.122904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517923e3-b818-4ec0-ae15-360d0a98de60 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.123328 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517923e3-b818-4ec0-ae15-360d0a98de60 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.123757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517923e3-b818-4ec0-ae15-360d0a98de60 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.134902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517923e3-b818-4ec0-ae15-360d0a98de60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:41.140066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-517923e3-b818-4ec0-ae15-360d0a98de60 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:41.140066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:41.140066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:41.140066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:41.140066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:41.140066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:41.140066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:41.140066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:41.140066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:41.140066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:41.140066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:41.140066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:41.140066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:41.140066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:41.140066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:41.140066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:41.141123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:41.141123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:41.141123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:41.141123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:41.141123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:41.141123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:41.141123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:41.141123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:41.141123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:41.141123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:41.141123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:41.141123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:41.141123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:41.141123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:41.141123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:41.141123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:41.142097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:41.142097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:41.142097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:41.142097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:41.142097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:41.142097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:41.142097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:41.142097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:41.142097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:41.142097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:41.142097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:41.142097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:04:41.144050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 35/72] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:41.145018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517923e3-b818-4ec0-ae15-360d0a98de60 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.164201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:41.169281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:41.215335 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853398e7-991b-4424-87d1-5659a49f2029 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.252410 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38372a-884d-424b-aa9e-5bcf1310e19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.253128 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38372a-884d-424b-aa9e-5bcf1310e19f None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.253388 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38372a-884d-424b-aa9e-5bcf1310e19f None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.253609 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38372a-884d-424b-aa9e-5bcf1310e19f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.265246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38372a-884d-424b-aa9e-5bcf1310e19f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:41.270631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-4f38372a-884d-424b-aa9e-5bcf1310e19f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:41.270631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:41.270631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:41.270631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:41.270631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:41.270631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:41.270631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:41.270631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:41.270631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:41.270631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:41.270631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:41.270631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:41.270631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:41.270631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:41.270631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:41.270631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:41.271565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:41.271565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:41.271565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:41.271565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:41.271565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:41.271565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:41.271565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:41.271565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:41.271565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:41.271565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:41.271565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:41.271565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:41.271565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:41.271565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:41.271565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:41.271565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:41.273063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:41.273063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:41.273063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:41.273063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:41.273063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:41.273063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:41.273063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:41.273063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:41.273063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:41.273063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:41.273063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:41.273063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:04:41.273063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 38/73] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:41.274733 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38372a-884d-424b-aa9e-5bcf1310e19f None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.291147 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517923e3-b818-4ec0-ae15-360d0a98de60 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.325826 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1c0d4-5ef0-4c65-a4f1-2fde5e95e380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.326383 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1c0d4-5ef0-4c65-a4f1-2fde5e95e380 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.326670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1c0d4-5ef0-4c65-a4f1-2fde5e95e380 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.326967 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1c0d4-5ef0-4c65-a4f1-2fde5e95e380 None admin] PATH_INFO: `/v3/users/7404abca54674994bf5e6ec8f024692d` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.341595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1c0d4-5ef0-4c65-a4f1-2fde5e95e380 None admin] RBAC: Authorizing `identity:get_user(user_id=7404abca54674994bf5e6ec8f024692d)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:41.343119 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1c0d4-5ef0-4c65-a4f1-2fde5e95e380 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:41.344959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 36/74] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/users/7404abca54674994bf5e6ec8f024692d => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:41.347740 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1c0d4-5ef0-4c65-a4f1-2fde5e95e380 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.383139 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38372a-884d-424b-aa9e-5bcf1310e19f None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.408994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:41.418260 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e578d2-49bf-4dc6-9f63-be204e5dd975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.418982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e578d2-49bf-4dc6-9f63-be204e5dd975 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.419274 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e578d2-49bf-4dc6-9f63-be204e5dd975 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.419559 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e578d2-49bf-4dc6-9f63-be204e5dd975 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.420510 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:41.430798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e578d2-49bf-4dc6-9f63-be204e5dd975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:41.436109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-32e578d2-49bf-4dc6-9f63-be204e5dd975 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:41.436109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:41.436109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:41.436109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:41.436109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:41.436109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:41.436109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:41.436109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:41.436109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:41.436109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:41.436109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:41.436109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:41.436109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:41.436109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:41.436109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:41.436109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:41.437238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:41.437238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:41.437238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:41.437238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:41.437238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:41.437238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:41.437238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:41.437238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:41.437238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:41.437238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:41.437238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:41.437238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:41.437238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:41.437238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:41.437238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:41.437238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:41.438112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:41.438112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:41.438112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:41.438112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:41.438112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:41.438112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:41.438112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:41.438112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:41.438112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:41.438112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:41.438112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:41.438112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:04:41.438112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 39/75] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:41.439025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87290dc9-668d-4666-bb5c-169f3961e9c6 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.439328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87290dc9-668d-4666-bb5c-169f3961e9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.440126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87290dc9-668d-4666-bb5c-169f3961e9c6 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.441564 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 40/76] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:41.442088 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87290dc9-668d-4666-bb5c-169f3961e9c6 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.457637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1c0d4-5ef0-4c65-a4f1-2fde5e95e380 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.497626 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb813612-0144-4682-af0b-abef5bef67fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.498307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb813612-0144-4682-af0b-abef5bef67fa None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.498525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb813612-0144-4682-af0b-abef5bef67fa None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.498798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb813612-0144-4682-af0b-abef5bef67fa None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.499505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb813612-0144-4682-af0b-abef5bef67fa None admin] RBAC: Adding query filter params (name=member) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:41.499809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb813612-0144-4682-af0b-abef5bef67fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:41.501210 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb813612-0144-4682-af0b-abef5bef67fa None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:41.510038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 37/77] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:41.510852 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb813612-0144-4682-af0b-abef5bef67fa None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.552098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87290dc9-668d-4666-bb5c-169f3961e9c6 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.586972 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d3c96-654a-4ed9-8803-76db12fc43e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.587575 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d3c96-654a-4ed9-8803-76db12fc43e4 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.587856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d3c96-654a-4ed9-8803-76db12fc43e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.588130 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d3c96-654a-4ed9-8803-76db12fc43e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.588828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d3c96-654a-4ed9-8803-76db12fc43e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:41.589135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d3c96-654a-4ed9-8803-76db12fc43e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:41.590543 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d3c96-654a-4ed9-8803-76db12fc43e4 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:41.599386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 41/78] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:41.600995 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eae9e5-47f4-449c-9e8c-cfd4f1d968cf None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.601281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eae9e5-47f4-449c-9e8c-cfd4f1d968cf None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.601560 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eae9e5-47f4-449c-9e8c-cfd4f1d968cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.619507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-c9eae9e5-47f4-449c-9e8c-cfd4f1d968cf None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:41.620669 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb813612-0144-4682-af0b-abef5bef67fa None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.620776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c9eae9e5-47f4-449c-9e8c-cfd4f1d968cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:41.655925 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760dfa4-ed15-4af1-bf28-4ce19ced7fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.656545 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760dfa4-ed15-4af1-bf28-4ce19ced7fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.656827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760dfa4-ed15-4af1-bf28-4ce19ced7fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.657105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760dfa4-ed15-4af1-bf28-4ce19ced7fd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.659827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:41.667307 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:41.690681 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 42/79] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:41.691574 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eae9e5-47f4-449c-9e8c-cfd4f1d968cf None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.738213 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760dfa4-ed15-4af1-bf28-4ce19ced7fd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:41.739992 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760dfa4-ed15-4af1-bf28-4ce19ced7fd4 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:41.832504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eae9e5-47f4-449c-9e8c-cfd4f1d968cf None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.859810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 38/80] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:41.860326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760dfa4-ed15-4af1-bf28-4ce19ced7fd4 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.876192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c2c0d-4d87-432a-9148-d933b871efe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.876820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c2c0d-4d87-432a-9148-d933b871efe9 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.877110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c2c0d-4d87-432a-9148-d933b871efe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.877384 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c2c0d-4d87-432a-9148-d933b871efe9 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.878066 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c2c0d-4d87-432a-9148-d933b871efe9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:41.878466 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c2c0d-4d87-432a-9148-d933b871efe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:41.879913 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c2c0d-4d87-432a-9148-d933b871efe9 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:41.888942 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 43/81] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:41.889587 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c2c0d-4d87-432a-9148-d933b871efe9 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.904580 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:41.937879 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:41.981455 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760dfa4-ed15-4af1-bf28-4ce19ced7fd4 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.011340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c2c0d-4d87-432a-9148-d933b871efe9 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.016682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05c0fa-3916-4818-8496-48b17fd8dbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.017325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05c0fa-3916-4818-8496-48b17fd8dbdc None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.017612 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05c0fa-3916-4818-8496-48b17fd8dbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.017892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05c0fa-3916-4818-8496-48b17fd8dbdc None admin] PATH_INFO: `/v3/users/7404abca54674994bf5e6ec8f024692d` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.019067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05c0fa-3916-4818-8496-48b17fd8dbdc None admin] RBAC: Authorizing `identity:get_user(user_id=7404abca54674994bf5e6ec8f024692d)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.020602 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05c0fa-3916-4818-8496-48b17fd8dbdc None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:42.022922 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 39/82] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/users/7404abca54674994bf5e6ec8f024692d => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:42.023910 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05c0fa-3916-4818-8496-48b17fd8dbdc None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:42.052005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3af79-7561-4f2c-b26c-f5143ec98204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.052644 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3af79-7561-4f2c-b26c-f5143ec98204 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.052922 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3af79-7561-4f2c-b26c-f5143ec98204 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.053199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3af79-7561-4f2c-b26c-f5143ec98204 None admin] PATH_INFO: `/v3/users/30d09a6b4f0047b4ba7decc0e37bf0cf` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.054442 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3af79-7561-4f2c-b26c-f5143ec98204 None admin] RBAC: Authorizing `identity:get_user(user_id=30d09a6b4f0047b4ba7decc0e37bf0cf)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.055925 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3af79-7561-4f2c-b26c-f5143ec98204 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:42.057488 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 44/83] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/users/30d09a6b4f0047b4ba7decc0e37bf0cf => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:42.058245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3af79-7561-4f2c-b26c-f5143ec98204 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:42.136396 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05c0fa-3916-4818-8496-48b17fd8dbdc None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.137881 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:42.169285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3af79-7561-4f2c-b26c-f5143ec98204 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.173804 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32140628-d493-478f-bd9e-ec9ec160d80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.174457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32140628-d493-478f-bd9e-ec9ec160d80d None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.174740 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32140628-d493-478f-bd9e-ec9ec160d80d None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.175055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32140628-d493-478f-bd9e-ec9ec160d80d None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.176291 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32140628-d493-478f-bd9e-ec9ec160d80d None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.177697 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32140628-d493-478f-bd9e-ec9ec160d80d None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:42.179087 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:42.179738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 40/84] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:42 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:42.180537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32140628-d493-478f-bd9e-ec9ec160d80d None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:42.211158 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7359c41-2109-4cf8-84c5-09c1770f8f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.212281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7359c41-2109-4cf8-84c5-09c1770f8f3c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.212632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7359c41-2109-4cf8-84c5-09c1770f8f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.213096 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7359c41-2109-4cf8-84c5-09c1770f8f3c None admin] PATH_INFO: `/v3/users/7404abca54674994bf5e6ec8f024692d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.220950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7359c41-2109-4cf8-84c5-09c1770f8f3c None admin] RBAC: Authorizing `identity:get_user(user_id=7404abca54674994bf5e6ec8f024692d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.223469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7359c41-2109-4cf8-84c5-09c1770f8f3c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:42.230214 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 45/85] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:42 2022] GET /identity/v3/users/7404abca54674994bf5e6ec8f024692d => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:42.231048 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7359c41-2109-4cf8-84c5-09c1770f8f3c None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:42.300043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32140628-d493-478f-bd9e-ec9ec160d80d None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.336753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e8439-1e1a-482c-ad5a-1fe415476003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.337407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e8439-1e1a-482c-ad5a-1fe415476003 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.337692 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e8439-1e1a-482c-ad5a-1fe415476003 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.337972 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e8439-1e1a-482c-ad5a-1fe415476003 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.339158 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e8439-1e1a-482c-ad5a-1fe415476003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.340645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e8439-1e1a-482c-ad5a-1fe415476003 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:42.342239 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 41/86] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:42.343051 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e8439-1e1a-482c-ad5a-1fe415476003 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:42.349505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7359c41-2109-4cf8-84c5-09c1770f8f3c None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.386253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:42.387391 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec5ea4-37d7-4c11-9673-b7e5d7ab4125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.387964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec5ea4-37d7-4c11-9673-b7e5d7ab4125 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.388248 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec5ea4-37d7-4c11-9673-b7e5d7ab4125 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.388528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec5ea4-37d7-4c11-9673-b7e5d7ab4125 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.442943 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:42.448453 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e8439-1e1a-482c-ad5a-1fe415476003 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.466030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec5ea4-37d7-4c11-9673-b7e5d7ab4125 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.467653 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec5ea4-37d7-4c11-9673-b7e5d7ab4125 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:42.484826 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a860d59-936e-4bd4-b334-11eb8afe6656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.485448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a860d59-936e-4bd4-b334-11eb8afe6656 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.485734 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a860d59-936e-4bd4-b334-11eb8afe6656 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.486023 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a860d59-936e-4bd4-b334-11eb8afe6656 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.557393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a860d59-936e-4bd4-b334-11eb8afe6656 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.558903 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a860d59-936e-4bd4-b334-11eb8afe6656 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:42.571990 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 46/87] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:42.576791 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec5ea4-37d7-4c11-9673-b7e5d7ab4125 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:42.599620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:42.656442 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:42.668392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 42/88] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:42.669302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a860d59-936e-4bd4-b334-11eb8afe6656 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:42.686163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec5ea4-37d7-4c11-9673-b7e5d7ab4125 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.727403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba84c7-57a8-4620-a84e-404fd7563b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.728316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba84c7-57a8-4620-a84e-404fd7563b67 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.728761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba84c7-57a8-4620-a84e-404fd7563b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.729039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba84c7-57a8-4620-a84e-404fd7563b67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.729789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba84c7-57a8-4620-a84e-404fd7563b67 None admin] RBAC: Adding query filter params (role.id=357fdcf3417c45319b9d424a5ab4e9ca, scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69, user.id=7404abca54674994bf5e6ec8f024692d) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:42.730110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba84c7-57a8-4620-a84e-404fd7563b67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.731626 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba84c7-57a8-4620-a84e-404fd7563b67 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:42.737707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 47/89] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:42 2022] GET /identity/v3/role_assignments?user.id=7404abca54674994bf5e6ec8f024692d&scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69&role.id=357fdcf3417c45319b9d424a5ab4e9ca => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:42.740139 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba84c7-57a8-4620-a84e-404fd7563b67 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:42.812067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a860d59-936e-4bd4-b334-11eb8afe6656 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.856487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a5e1d-1014-44ef-aa06-e3659a0cb90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.857348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a5e1d-1014-44ef-aa06-e3659a0cb90e None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.857642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a5e1d-1014-44ef-aa06-e3659a0cb90e None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.858012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a5e1d-1014-44ef-aa06-e3659a0cb90e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.871258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:42.889250 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba84c7-57a8-4620-a84e-404fd7563b67 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.934511 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16753c36-fe85-4f0c-b23f-33100040e826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.935775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16753c36-fe85-4f0c-b23f-33100040e826 None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.935775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16753c36-fe85-4f0c-b23f-33100040e826 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.935775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16753c36-fe85-4f0c-b23f-33100040e826 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.936313 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16753c36-fe85-4f0c-b23f-33100040e826 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.937777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16753c36-fe85-4f0c-b23f-33100040e826 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:42.952657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a5e1d-1014-44ef-aa06-e3659a0cb90e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.952915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:42.954146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a5e1d-1014-44ef-aa06-e3659a0cb90e None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:43.051029 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 48/90] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:04:42 2022] POST /identity/v3/users => generated 289 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:04:43.060074 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16753c36-fe85-4f0c-b23f-33100040e826 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:43.081907 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 43/91] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:43.083112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a5e1d-1014-44ef-aa06-e3659a0cb90e None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:43.155557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:43.237599 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16753c36-fe85-4f0c-b23f-33100040e826 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:43.249756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a5e1d-1014-44ef-aa06-e3659a0cb90e None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:43.298295 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406a968-9920-42a2-a197-2991b222bc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:43.299512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406a968-9920-42a2-a197-2991b222bc82 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:43.300400 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406a968-9920-42a2-a197-2991b222bc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:43.301074 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406a968-9920-42a2-a197-2991b222bc82 None admin] PATH_INFO: `/v3/projects/96a275e837304a4f81ef4efa38975cca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:43.302781 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406a968-9920-42a2-a197-2991b222bc82 None admin] RBAC: Authorizing `identity:get_project(project_id=96a275e837304a4f81ef4efa38975cca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:43.305772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406a968-9920-42a2-a197-2991b222bc82 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:43.307923 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 49/92] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:43 2022] GET /identity/v3/projects/96a275e837304a4f81ef4efa38975cca => generated 312 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:43.308896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406a968-9920-42a2-a197-2991b222bc82 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:43.317355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c6238-944f-42e6-af9c-487e2e8da8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:43.318157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c6238-944f-42e6-af9c-487e2e8da8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:43.318550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c6238-944f-42e6-af9c-487e2e8da8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:43.320213 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c6238-944f-42e6-af9c-487e2e8da8e6 None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:43.321446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c6238-944f-42e6-af9c-487e2e8da8e6 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:43.323105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c6238-944f-42e6-af9c-487e2e8da8e6 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:43.324957 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 44/93] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:43 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:43.326534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c6238-944f-42e6-af9c-487e2e8da8e6 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:43.401031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:43.446788 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406a968-9920-42a2-a197-2991b222bc82 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:43.484343 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c6238-944f-42e6-af9c-487e2e8da8e6 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:43.487245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2db8f-e68b-4e9f-9c97-aef162b2cfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:43.488533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2db8f-e68b-4e9f-9c97-aef162b2cfdf None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:43.488533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2db8f-e68b-4e9f-9c97-aef162b2cfdf None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:43.488533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2db8f-e68b-4e9f-9c97-aef162b2cfdf None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:43.489897 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f2db8f-e68b-4e9f-9c97-aef162b2cfdf None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:43.491850 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f2db8f-e68b-4e9f-9c97-aef162b2cfdf None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:43.493255 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 50/94] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:43 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:43.494049 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2db8f-e68b-4e9f-9c97-aef162b2cfdf None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:43.495883 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:43.534117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd464cc4-af10-4b70-ba49-b486de3f5697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:43.535028 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd464cc4-af10-4b70-ba49-b486de3f5697 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:43.535314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd464cc4-af10-4b70-ba49-b486de3f5697 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:43.535590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd464cc4-af10-4b70-ba49-b486de3f5697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:43.536774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd464cc4-af10-4b70-ba49-b486de3f5697 None admin] RBAC: Adding query filter params (role.id=357fdcf3417c45319b9d424a5ab4e9ca, scope.project.id=96a275e837304a4f81ef4efa38975cca, user.id=7404abca54674994bf5e6ec8f024692d) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:43.537086 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd464cc4-af10-4b70-ba49-b486de3f5697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:43.540368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd464cc4-af10-4b70-ba49-b486de3f5697 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:43.548947 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 45/95] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:43 2022] GET /identity/v3/role_assignments?user.id=7404abca54674994bf5e6ec8f024692d&scope.project.id=96a275e837304a4f81ef4efa38975cca&role.id=357fdcf3417c45319b9d424a5ab4e9ca => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:43.549910 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd464cc4-af10-4b70-ba49-b486de3f5697 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:43.653386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2db8f-e68b-4e9f-9c97-aef162b2cfdf None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:43.721571 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:43.736699 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550518a3-0e32-49f7-84bd-1eaea2914c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:43.737337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550518a3-0e32-49f7-84bd-1eaea2914c84 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:43.737621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550518a3-0e32-49f7-84bd-1eaea2914c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:43.737900 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550518a3-0e32-49f7-84bd-1eaea2914c84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:43.738700 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550518a3-0e32-49f7-84bd-1eaea2914c84 None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69, user.id=30d09a6b4f0047b4ba7decc0e37bf0cf) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:43.738987 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550518a3-0e32-49f7-84bd-1eaea2914c84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:43.740753 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550518a3-0e32-49f7-84bd-1eaea2914c84 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:43.754026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 51/96] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:43 2022] GET /identity/v3/role_assignments?user.id=30d09a6b4f0047b4ba7decc0e37bf0cf&scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:43.806276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd464cc4-af10-4b70-ba49-b486de3f5697 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:43.855296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a745512-47fa-461d-9d8a-f1552254ec73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:43.856000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a745512-47fa-461d-9d8a-f1552254ec73 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:43.856293 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a745512-47fa-461d-9d8a-f1552254ec73 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:43.856579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a745512-47fa-461d-9d8a-f1552254ec73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:43.857333 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a745512-47fa-461d-9d8a-f1552254ec73 None admin] RBAC: Adding query filter params (role.id=e14f37f4a3a24f8da0f757aabb1f39be, scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69, user.id=7404abca54674994bf5e6ec8f024692d) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:43.857651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a745512-47fa-461d-9d8a-f1552254ec73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:43.859144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a745512-47fa-461d-9d8a-f1552254ec73 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:43.870980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 46/97] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:43 2022] GET /identity/v3/role_assignments?user.id=7404abca54674994bf5e6ec8f024692d&scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69&role.id=e14f37f4a3a24f8da0f757aabb1f39be => generated 259 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:45.262603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca95134-038b-4c68-a120-46f7ab4e760f None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:45.263149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca95134-038b-4c68-a120-46f7ab4e760f None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:45.263223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca95134-038b-4c68-a120-46f7ab4e760f None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:45.264545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 52/98] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:45.276250 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61aa467-5337-4768-b155-f9f37946b192 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:45.277031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61aa467-5337-4768-b155-f9f37946b192 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:45.277591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61aa467-5337-4768-b155-f9f37946b192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:45.311913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-c61aa467-5337-4768-b155-f9f37946b192 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:45.313446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c61aa467-5337-4768-b155-f9f37946b192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:45.349645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:45.409229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 47/99] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:45.956811 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca95134-038b-4c68-a120-46f7ab4e760f None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:46.023250 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae25a43-221b-446d-a5d8-08bbb0e39666 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:46.023914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae25a43-221b-446d-a5d8-08bbb0e39666 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:46.024387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae25a43-221b-446d-a5d8-08bbb0e39666 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:46.026007 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 48/100] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:04:46.032037 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caef024-1aaf-467d-9bcf-29d2b9135826 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:46.032571 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caef024-1aaf-467d-9bcf-29d2b9135826 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:46.033062 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caef024-1aaf-467d-9bcf-29d2b9135826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:46.064679 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-3caef024-1aaf-467d-9bcf-29d2b9135826 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:46.066242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3caef024-1aaf-467d-9bcf-29d2b9135826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:46.104422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca95134-038b-4c68-a120-46f7ab4e760f None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:46.162264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60b62c-de4d-4eee-a11c-27fdf110af56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:46.163058 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60b62c-de4d-4eee-a11c-27fdf110af56 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:46.163343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60b62c-de4d-4eee-a11c-27fdf110af56 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:46.163673 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60b62c-de4d-4eee-a11c-27fdf110af56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:46.185320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60b62c-de4d-4eee-a11c-27fdf110af56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:46.190727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-dd60b62c-de4d-4eee-a11c-27fdf110af56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:46.190727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:46.190727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:46.190727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:46.190727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:46.190727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:46.190727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:46.190727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:46.190727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:46.190727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:46.190727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:46.190727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:46.190727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:46.190727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:46.190727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:46.190727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:46.191613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:46.191613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:46.191613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:46.191613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:46.191613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:46.191613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:46.191613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:46.191613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:46.191613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:46.191613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:46.191613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:46.191613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:46.191613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:46.191613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:46.191613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:46.191613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:46.192386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:46.192386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:46.192386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:46.192386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:46.192386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:46.192386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:46.192386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:46.192386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:46.192386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:46.192386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:46.192386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:46.192386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:04:46.192386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 53/101] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:46.193028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e8b5c-cc55-47e1-a4f1-70ce590b1a2d None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:46.193569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e8b5c-cc55-47e1-a4f1-70ce590b1a2d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:46.193569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e8b5c-cc55-47e1-a4f1-70ce590b1a2d None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:46.196764 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 54/102] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:46.200346 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e8b5c-cc55-47e1-a4f1-70ce590b1a2d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:46.265289 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:46.266009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 49/103] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:46.267467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dea66-d44e-429f-a919-798f445e46b2 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:46.267753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dea66-d44e-429f-a919-798f445e46b2 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:46.268043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dea66-d44e-429f-a919-798f445e46b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:46.363917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-3e0dea66-d44e-429f-a919-798f445e46b2 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:46.365382 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3e0dea66-d44e-429f-a919-798f445e46b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:46.396584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e8b5c-cc55-47e1-a4f1-70ce590b1a2d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:46.434631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b1d90-116d-4957-9a0f-9189be4d1483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:46.435291 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b1d90-116d-4957-9a0f-9189be4d1483 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:46.435575 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b1d90-116d-4957-9a0f-9189be4d1483 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:46.435855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b1d90-116d-4957-9a0f-9189be4d1483 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:46.436537 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b1d90-116d-4957-9a0f-9189be4d1483 None admin] RBAC: Adding query filter params (name=member) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:46.436843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b1d90-116d-4957-9a0f-9189be4d1483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:46.438253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b1d90-116d-4957-9a0f-9189be4d1483 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:46.448666 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 55/104] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:04:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:46.454245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b1d90-116d-4957-9a0f-9189be4d1483 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:46.525317 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 50/105] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:46.581623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b1d90-116d-4957-9a0f-9189be4d1483 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:46.606572 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688869f3-ae4e-41d7-a67c-b01dc2357beb None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:46.606906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688869f3-ae4e-41d7-a67c-b01dc2357beb None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:46.607175 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688869f3-ae4e-41d7-a67c-b01dc2357beb None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:46.608671 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 51/106] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:04:46.609833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:46.615916 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb5515-7193-4356-87d4-cf6c841cd41a None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:46.616505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb5515-7193-4356-87d4-cf6c841cd41a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:46.616999 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb5515-7193-4356-87d4-cf6c841cd41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:46.637227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-fbbb5515-7193-4356-87d4-cf6c841cd41a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:46.639035 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-fbbb5515-7193-4356-87d4-cf6c841cd41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:46.639159 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf0133-6161-4914-bbf3-15445d1ff566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:46.639733 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf0133-6161-4914-bbf3-15445d1ff566 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:46.640076 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf0133-6161-4914-bbf3-15445d1ff566 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:46.640366 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf0133-6161-4914-bbf3-15445d1ff566 None admin] PATH_INFO: `/v3/users/7404abca54674994bf5e6ec8f024692d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:46.642801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf0133-6161-4914-bbf3-15445d1ff566 None admin] RBAC: Authorizing `identity:get_user(user_id=7404abca54674994bf5e6ec8f024692d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:46.644626 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf0133-6161-4914-bbf3-15445d1ff566 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:46.646493 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 56/107] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:46 2022] GET /identity/v3/users/7404abca54674994bf5e6ec8f024692d => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:46.653476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf0133-6161-4914-bbf3-15445d1ff566 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:46.668092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:46.769806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 52/108] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:46.770898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb5515-7193-4356-87d4-cf6c841cd41a None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:46.848277 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf0133-6161-4914-bbf3-15445d1ff566 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:46.899332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4d6ad-823a-4777-8c1a-0200edefab5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:46.899980 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4d6ad-823a-4777-8c1a-0200edefab5c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:46.900264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4d6ad-823a-4777-8c1a-0200edefab5c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:46.900542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4d6ad-823a-4777-8c1a-0200edefab5c None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:46.901852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4d6ad-823a-4777-8c1a-0200edefab5c None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:46.903500 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4d6ad-823a-4777-8c1a-0200edefab5c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:46.905407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 57/109] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:46 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:46.906376 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9eec35-7e73-4aef-b6b5-3ac8e6d8c92e None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:46.906665 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9eec35-7e73-4aef-b6b5-3ac8e6d8c92e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:46.911887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9eec35-7e73-4aef-b6b5-3ac8e6d8c92e None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:46.914332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 58/110] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:46.915090 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9eec35-7e73-4aef-b6b5-3ac8e6d8c92e None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:46.931099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb5515-7193-4356-87d4-cf6c841cd41a None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:46.970060 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9916a-8dc5-4bc9-b21a-0b2b3a1fe7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:46.970714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9916a-8dc5-4bc9-b21a-0b2b3a1fe7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:46.971080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9916a-8dc5-4bc9-b21a-0b2b3a1fe7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:46.971363 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9916a-8dc5-4bc9-b21a-0b2b3a1fe7c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:46.972495 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9916a-8dc5-4bc9-b21a-0b2b3a1fe7c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:46.973992 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9916a-8dc5-4bc9-b21a-0b2b3a1fe7c3 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:46.976585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 53/111] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:46.977359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9916a-8dc5-4bc9-b21a-0b2b3a1fe7c3 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:46.990352 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:47.005198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:47.043566 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9eec35-7e73-4aef-b6b5-3ac8e6d8c92e None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:47.081306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf40ecc-fd3e-4cab-b3e1-b86342980229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:47.081952 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf40ecc-fd3e-4cab-b3e1-b86342980229 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.082240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf40ecc-fd3e-4cab-b3e1-b86342980229 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.082517 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf40ecc-fd3e-4cab-b3e1-b86342980229 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.086217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9916a-8dc5-4bc9-b21a-0b2b3a1fe7c3 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:47.093320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf40ecc-fd3e-4cab-b3e1-b86342980229 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:47.098193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-7cf40ecc-fd3e-4cab-b3e1-b86342980229 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:04:47.098193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:47.098193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:47.098193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:47.098193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:47.098193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:47.098193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:47.098193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:47.098193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:47.098193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:47.098193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:47.098193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:47.098193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:47.098193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:47.098193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:47.098193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:47.099392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:47.099392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:47.099392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:47.099392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:47.099392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:47.099392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:47.099392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:47.099392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:47.099392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:47.099392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:47.099392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:47.099392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:47.099392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:47.099392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:47.099392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:47.099392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:47.101441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:47.101441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:47.101441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:47.101441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:47.101441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:47.101441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:47.101441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:47.101441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:47.101441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:47.101441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:47.101441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:04:47.101441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:04:47.101441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 59/112] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:47.101441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16ad47-a775-4a92-8740-707cfc0510d6 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.102346 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16ad47-a775-4a92-8740-707cfc0510d6 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.102346 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16ad47-a775-4a92-8740-707cfc0510d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.120725 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-cd16ad47-a775-4a92-8740-707cfc0510d6 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:47.121980 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-cd16ad47-a775-4a92-8740-707cfc0510d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:47.128261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc96d7-ad47-47ac-a365-63d7c279ff3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:47.128923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc96d7-ad47-47ac-a365-63d7c279ff3b None admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.129206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc96d7-ad47-47ac-a365-63d7c279ff3b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.129484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc96d7-ad47-47ac-a365-63d7c279ff3b None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69/users/7404abca54674994bf5e6ec8f024692d/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.131514 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc96d7-ad47-47ac-a365-63d7c279ff3b None admin] RBAC: Authorizing `identity:create_grant(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69, user_id=7404abca54674994bf5e6ec8f024692d, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:47.132960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc96d7-ad47-47ac-a365-63d7c279ff3b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:47.187642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 54/113] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:04:46 2022] PUT /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69/users/7404abca54674994bf5e6ec8f024692d/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:47.188915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef183c9-0311-44f9-b8f6-23e24f81c8c4 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.189165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef183c9-0311-44f9-b8f6-23e24f81c8c4 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.189462 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef183c9-0311-44f9-b8f6-23e24f81c8c4 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.190728 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 55/114] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:47.191481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef183c9-0311-44f9-b8f6-23e24f81c8c4 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:47.198225 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 60/115] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:47.199597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16ad47-a775-4a92-8740-707cfc0510d6 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:47.286036 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:47.321176 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:47.352676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef183c9-0311-44f9-b8f6-23e24f81c8c4 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:47.371380 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16ad47-a775-4a92-8740-707cfc0510d6 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:47.398118 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d9190-e528-4fd4-b4b7-634319b61ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:47.398886 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9190-e528-4fd4-b4b7-634319b61ce9 None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.399260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9190-e528-4fd4-b4b7-634319b61ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.399791 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9190-e528-4fd4-b4b7-634319b61ce9 None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.400920 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d9190-e528-4fd4-b4b7-634319b61ce9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:47.403005 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d9190-e528-4fd4-b4b7-634319b61ce9 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:47.412156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afa37b-172b-420b-a265-588fc6952e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:47.413582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afa37b-172b-420b-a265-588fc6952e7f None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.413869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afa37b-172b-420b-a265-588fc6952e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.414142 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afa37b-172b-420b-a265-588fc6952e7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.433007 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24afa37b-172b-420b-a265-588fc6952e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:47.438515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-24afa37b-172b-420b-a265-588fc6952e7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:47.438515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:47.438515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:47.438515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:47.438515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:47.438515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:47.438515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:47.438515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:47.438515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:47.438515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:47.438515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:47.438515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:47.438515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:47.438515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:47.438515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:47.438515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:47.446340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:47.446340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:47.446340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:47.446340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:47.446340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:47.446340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:47.446340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:47.446340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:47.446340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:47.446340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:47.446340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:47.446340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:47.446340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:47.446340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:47.446340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:47.446340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:47.447209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:47.447209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:47.447209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:47.447209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:47.447209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:47.447209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:47.447209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:47.447209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:47.447209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:47.447209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:47.447209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:47.447209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:04:47.447209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 61/116] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:47.447209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afa37b-172b-420b-a265-588fc6952e7f None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:47.498201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 56/117] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 31 19:04:47 2022] POST /identity/v3/projects => generated 302 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:04:47.500794 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1687f9-9740-4b6f-ab7e-a9ce0d25663e None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.501338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1687f9-9740-4b6f-ab7e-a9ce0d25663e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.501826 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1687f9-9740-4b6f-ab7e-a9ce0d25663e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.532565 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-cf1687f9-9740-4b6f-ab7e-a9ce0d25663e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:47.535069 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-cf1687f9-9740-4b6f-ab7e-a9ce0d25663e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:47.564900 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afa37b-172b-420b-a265-588fc6952e7f None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:47.569191 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:47.613277 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731714b-8c08-4148-8f8f-ff5fb4073e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:47.614842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731714b-8c08-4148-8f8f-ff5fb4073e55 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.616477 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731714b-8c08-4148-8f8f-ff5fb4073e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.617129 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731714b-8c08-4148-8f8f-ff5fb4073e55 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.618254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731714b-8c08-4148-8f8f-ff5fb4073e55 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:47.619598 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731714b-8c08-4148-8f8f-ff5fb4073e55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:47.622954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731714b-8c08-4148-8f8f-ff5fb4073e55 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:47.636579 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 62/118] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:04:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:47.637957 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731714b-8c08-4148-8f8f-ff5fb4073e55 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:47.638494 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 57/119] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:47.639529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1687f9-9740-4b6f-ab7e-a9ce0d25663e None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:47.742761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:47.773099 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731714b-8c08-4148-8f8f-ff5fb4073e55 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:47.781310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1687f9-9740-4b6f-ab7e-a9ce0d25663e None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:47.822816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6844a-4cde-4a60-b90d-b6c0127e8d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:47.823498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6844a-4cde-4a60-b90d-b6c0127e8d4e None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.823785 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6844a-4cde-4a60-b90d-b6c0127e8d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.824238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6844a-4cde-4a60-b90d-b6c0127e8d4e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.837483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6844a-4cde-4a60-b90d-b6c0127e8d4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:47.844000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-b2a6844a-4cde-4a60-b90d-b6c0127e8d4e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:47.844000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:47.844000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:47.844000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:47.844000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:47.844000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:47.844000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:47.844000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:47.844000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:47.844000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:47.844000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:47.844000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:47.844000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:47.844000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:47.844000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:47.844000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:47.845105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:47.845105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:47.845105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:47.845105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:47.845105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:47.845105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:47.845105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:47.845105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:47.845105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:47.845105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:47.845105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:47.845105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:47.845105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:47.845105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:47.845105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:47.845105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:47.846188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:47.846188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:47.846188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:47.846188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:47.846188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:47.846188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:47.846188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:47.846188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:47.846188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:47.846188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:47.846188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:47.846188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:04:47.846188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 58/120] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:04:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:47.846188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6844a-4cde-4a60-b90d-b6c0127e8d4e None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:47.857700 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adfea5-463b-4dfc-a5ca-032ef9b810df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:47.858329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adfea5-463b-4dfc-a5ca-032ef9b810df None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.858609 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adfea5-463b-4dfc-a5ca-032ef9b810df None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.858964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adfea5-463b-4dfc-a5ca-032ef9b810df None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.859643 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adfea5-463b-4dfc-a5ca-032ef9b810df None admin] RBAC: Adding query filter params (name=member) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:47.860006 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adfea5-463b-4dfc-a5ca-032ef9b810df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:47.861410 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adfea5-463b-4dfc-a5ca-032ef9b810df None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:47.880150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 63/121] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:04:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:47.880974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adfea5-463b-4dfc-a5ca-032ef9b810df None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:47.957486 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:47.995210 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6844a-4cde-4a60-b90d-b6c0127e8d4e None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:48.028078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:48.042112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13582c-a913-48ca-9b77-ff8ba7cfce89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:48.042767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13582c-a913-48ca-9b77-ff8ba7cfce89 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.043103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13582c-a913-48ca-9b77-ff8ba7cfce89 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.043384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13582c-a913-48ca-9b77-ff8ba7cfce89 None admin] PATH_INFO: `/v3/users/3932ff5e04dd4aadbc9ba0548512f2be` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.043641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adfea5-463b-4dfc-a5ca-032ef9b810df None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:48.060740 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13582c-a913-48ca-9b77-ff8ba7cfce89 None admin] RBAC: Authorizing `identity:get_user(user_id=3932ff5e04dd4aadbc9ba0548512f2be)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:48.062675 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13582c-a913-48ca-9b77-ff8ba7cfce89 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:48.064743 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 59/122] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:47 2022] GET /identity/v3/users/3932ff5e04dd4aadbc9ba0548512f2be => generated 289 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:48.065607 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13582c-a913-48ca-9b77-ff8ba7cfce89 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:48.088313 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0d328-ed0c-4118-9df8-e1adf00b27b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:48.089249 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0d328-ed0c-4118-9df8-e1adf00b27b0 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.089539 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0d328-ed0c-4118-9df8-e1adf00b27b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.089820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0d328-ed0c-4118-9df8-e1adf00b27b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.090502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0d328-ed0c-4118-9df8-e1adf00b27b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:48.090821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0d328-ed0c-4118-9df8-e1adf00b27b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:48.092230 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0d328-ed0c-4118-9df8-e1adf00b27b0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:48.103657 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 64/123] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:48.104648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0d328-ed0c-4118-9df8-e1adf00b27b0 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:48.185932 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13582c-a913-48ca-9b77-ff8ba7cfce89 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:48.221002 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:48.225579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad38cf-d192-4f4f-87e4-d307b82e42e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:48.232539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad38cf-d192-4f4f-87e4-d307b82e42e5 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.232539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad38cf-d192-4f4f-87e4-d307b82e42e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.232539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad38cf-d192-4f4f-87e4-d307b82e42e5 None admin] PATH_INFO: `/v3/users/7404abca54674994bf5e6ec8f024692d` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.232539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad38cf-d192-4f4f-87e4-d307b82e42e5 None admin] RBAC: Authorizing `identity:get_user(user_id=7404abca54674994bf5e6ec8f024692d)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:48.232539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad38cf-d192-4f4f-87e4-d307b82e42e5 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:48.232539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 60/124] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:48 2022] GET /identity/v3/users/7404abca54674994bf5e6ec8f024692d => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:48.233250 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0d328-ed0c-4118-9df8-e1adf00b27b0 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:48.236723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad38cf-d192-4f4f-87e4-d307b82e42e5 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:48.277842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0f492-e833-45e8-8eb5-e0acb5b71231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:48.278970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0f492-e833-45e8-8eb5-e0acb5b71231 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.279400 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0f492-e833-45e8-8eb5-e0acb5b71231 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.280736 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0f492-e833-45e8-8eb5-e0acb5b71231 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.298187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:48.304471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0f492-e833-45e8-8eb5-e0acb5b71231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:48.313039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-70f0f492-e833-45e8-8eb5-e0acb5b71231 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:48.313039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:48.313039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:48.313039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:48.313039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:48.313039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:48.313039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:48.313039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:48.313039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:48.313039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:48.313039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:48.313039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:48.313039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:48.313039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:48.313039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:48.313039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:48.314119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:48.314119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:48.314119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:48.314119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:48.314119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:48.314119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:48.314119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:48.314119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:48.314119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:48.314119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:48.314119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:48.314119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:48.314119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:48.314119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:48.314119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:48.314119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:48.315177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:48.315177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:48.315177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:48.315177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:48.315177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:48.315177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:48.315177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:48.315177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:48.315177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:48.315177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:48.315177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:48.315177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:04:48.317586 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 65/125] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:48.324203 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0f492-e833-45e8-8eb5-e0acb5b71231 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:48.363207 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad38cf-d192-4f4f-87e4-d307b82e42e5 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:48.406588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7568abc-690b-4ffe-8a87-0844e30fdfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:48.407852 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7568abc-690b-4ffe-8a87-0844e30fdfef None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.408242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7568abc-690b-4ffe-8a87-0844e30fdfef None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.408984 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7568abc-690b-4ffe-8a87-0844e30fdfef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.466854 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0f492-e833-45e8-8eb5-e0acb5b71231 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:48.508590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7568abc-690b-4ffe-8a87-0844e30fdfef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:48.510524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7568abc-690b-4ffe-8a87-0844e30fdfef None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:48.514310 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685971f-c160-49fa-a3bd-39edc9956343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:48.514986 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685971f-c160-49fa-a3bd-39edc9956343 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.515272 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685971f-c160-49fa-a3bd-39edc9956343 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.515544 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685971f-c160-49fa-a3bd-39edc9956343 None admin] PATH_INFO: `/v3/users/7404abca54674994bf5e6ec8f024692d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.516687 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685971f-c160-49fa-a3bd-39edc9956343 None admin] RBAC: Authorizing `identity:get_user(user_id=7404abca54674994bf5e6ec8f024692d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:48.518118 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685971f-c160-49fa-a3bd-39edc9956343 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:48.519732 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 66/126] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:48 2022] GET /identity/v3/users/7404abca54674994bf5e6ec8f024692d => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:48.524537 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685971f-c160-49fa-a3bd-39edc9956343 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:48.552668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:48.589408 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:48.652481 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685971f-c160-49fa-a3bd-39edc9956343 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:48.676316 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 61/127] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:48.676316 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7568abc-690b-4ffe-8a87-0844e30fdfef None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:48.704416 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ecb904-fca5-4661-9cfc-5c1e73f82d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:48.705656 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecb904-fca5-4661-9cfc-5c1e73f82d63 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.706411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecb904-fca5-4661-9cfc-5c1e73f82d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.707355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecb904-fca5-4661-9cfc-5c1e73f82d63 None admin] PATH_INFO: `/v3/projects/96a275e837304a4f81ef4efa38975cca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.709807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ecb904-fca5-4661-9cfc-5c1e73f82d63 None admin] RBAC: Authorizing `identity:get_project(project_id=96a275e837304a4f81ef4efa38975cca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:48.712372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ecb904-fca5-4661-9cfc-5c1e73f82d63 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:48.715401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 67/128] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:48 2022] GET /identity/v3/projects/96a275e837304a4f81ef4efa38975cca => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:48.720197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ecb904-fca5-4661-9cfc-5c1e73f82d63 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:48.795140 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7568abc-690b-4ffe-8a87-0844e30fdfef None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:48.842348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41482929-69c6-4eb2-8482-a78cd74824af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:48.843647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:48.844871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41482929-69c6-4eb2-8482-a78cd74824af None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.845365 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41482929-69c6-4eb2-8482-a78cd74824af None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.845865 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41482929-69c6-4eb2-8482-a78cd74824af None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.847383 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41482929-69c6-4eb2-8482-a78cd74824af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:48.847928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41482929-69c6-4eb2-8482-a78cd74824af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:48.850012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41482929-69c6-4eb2-8482-a78cd74824af None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:48.859976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ecb904-fca5-4661-9cfc-5c1e73f82d63 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:48.863531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 62/129] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:48.864512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41482929-69c6-4eb2-8482-a78cd74824af None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:48.887796 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:48.900929 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0d84c-a22b-4f05-a6da-3d809fcd90fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:48.901575 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0d84c-a22b-4f05-a6da-3d809fcd90fa None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.901856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0d84c-a22b-4f05-a6da-3d809fcd90fa None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.902196 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0d84c-a22b-4f05-a6da-3d809fcd90fa None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.903384 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0d84c-a22b-4f05-a6da-3d809fcd90fa None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:48.905313 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0d84c-a22b-4f05-a6da-3d809fcd90fa None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:48.907185 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 68/130] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:48 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:48.907759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0d84c-a22b-4f05-a6da-3d809fcd90fa None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:49.020138 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41482929-69c6-4eb2-8482-a78cd74824af None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:49.029451 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0d84c-a22b-4f05-a6da-3d809fcd90fa None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:49.068197 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a418f-42b1-40f0-aa1e-4516588f4098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:49.069150 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a418f-42b1-40f0-aa1e-4516588f4098 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.069611 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a418f-42b1-40f0-aa1e-4516588f4098 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.070051 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a418f-42b1-40f0-aa1e-4516588f4098 None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.071422 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a418f-42b1-40f0-aa1e-4516588f4098 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:49.072432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800439e-38a6-4262-8bf4-6d529f790bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:49.073200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800439e-38a6-4262-8bf4-6d529f790bd1 None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.073416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a418f-42b1-40f0-aa1e-4516588f4098 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:49.073665 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800439e-38a6-4262-8bf4-6d529f790bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.074040 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800439e-38a6-4262-8bf4-6d529f790bd1 None admin] PATH_INFO: `/v3/projects/96a275e837304a4f81ef4efa38975cca/users/7404abca54674994bf5e6ec8f024692d/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.076545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800439e-38a6-4262-8bf4-6d529f790bd1 None admin] RBAC: Authorizing `identity:create_grant(project_id=96a275e837304a4f81ef4efa38975cca, user_id=7404abca54674994bf5e6ec8f024692d, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:49.080788 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800439e-38a6-4262-8bf4-6d529f790bd1 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:49.081257 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 63/131] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:48 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:49.082312 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a418f-42b1-40f0-aa1e-4516588f4098 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:49.162504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 69/132] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:04:48 2022] PUT /identity/v3/projects/96a275e837304a4f81ef4efa38975cca/users/7404abca54674994bf5e6ec8f024692d/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:49.163790 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800439e-38a6-4262-8bf4-6d529f790bd1 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:49.207894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:49.216878 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:49.281113 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a418f-42b1-40f0-aa1e-4516588f4098 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:49.333482 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800439e-38a6-4262-8bf4-6d529f790bd1 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:49.358090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e4366-1ee1-4f1d-b14b-242963a6387d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:49.358944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e4366-1ee1-4f1d-b14b-242963a6387d None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.359360 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e4366-1ee1-4f1d-b14b-242963a6387d None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.360599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e4366-1ee1-4f1d-b14b-242963a6387d None admin] PATH_INFO: `/v3/users/30d09a6b4f0047b4ba7decc0e37bf0cf` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.361926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e4366-1ee1-4f1d-b14b-242963a6387d None admin] RBAC: Authorizing `identity:get_user(user_id=30d09a6b4f0047b4ba7decc0e37bf0cf)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:49.363593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e4366-1ee1-4f1d-b14b-242963a6387d None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:49.366293 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 64/133] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:49 2022] GET /identity/v3/users/30d09a6b4f0047b4ba7decc0e37bf0cf => generated 247 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:49.368512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e4366-1ee1-4f1d-b14b-242963a6387d None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:49.389943 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff4408-6544-42d0-8ec0-8ef5794210fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:49.390628 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff4408-6544-42d0-8ec0-8ef5794210fe None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.390936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff4408-6544-42d0-8ec0-8ef5794210fe None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.391219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff4408-6544-42d0-8ec0-8ef5794210fe None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69/users/7404abca54674994bf5e6ec8f024692d/roles/e14f37f4a3a24f8da0f757aabb1f39be` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.393504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff4408-6544-42d0-8ec0-8ef5794210fe None admin] RBAC: Authorizing `identity:create_grant(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69, user_id=7404abca54674994bf5e6ec8f024692d, role_id=e14f37f4a3a24f8da0f757aabb1f39be)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:49.395583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff4408-6544-42d0-8ec0-8ef5794210fe None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:49.456322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 70/134] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:04:49 2022] PUT /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69/users/7404abca54674994bf5e6ec8f024692d/roles/e14f37f4a3a24f8da0f757aabb1f39be => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:49.457185 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff4408-6544-42d0-8ec0-8ef5794210fe None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:49.529615 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e4366-1ee1-4f1d-b14b-242963a6387d None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:49.594653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfbf02-9850-47db-be61-a2181129e228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:49.595371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfbf02-9850-47db-be61-a2181129e228 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.595657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfbf02-9850-47db-be61-a2181129e228 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.595931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfbf02-9850-47db-be61-a2181129e228 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.596687 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfbf02-9850-47db-be61-a2181129e228 None admin] RBAC: Adding query filter params (role.id=6c1ffb941f1646d2bf707005c7796790, scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69, user.id=3932ff5e04dd4aadbc9ba0548512f2be) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:49.597001 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfbf02-9850-47db-be61-a2181129e228 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:49.598482 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfbf02-9850-47db-be61-a2181129e228 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:49.617099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 65/135] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:49 2022] GET /identity/v3/role_assignments?user.id=3932ff5e04dd4aadbc9ba0548512f2be&scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69&role.id=6c1ffb941f1646d2bf707005c7796790 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:49.620469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca2a74-fed8-424d-89f6-3f803aeb89ca None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.621022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca2a74-fed8-424d-89f6-3f803aeb89ca None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.621593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca2a74-fed8-424d-89f6-3f803aeb89ca None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.623469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 66/136] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:49.623958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:49.630213 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fbc9a-d8ec-4d80-af23-fb102455d386 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.630554 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fbc9a-d8ec-4d80-af23-fb102455d386 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.630839 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fbc9a-d8ec-4d80-af23-fb102455d386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.676112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-692fbc9a-d8ec-4d80-af23-fb102455d386 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:49.677384 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff4408-6544-42d0-8ec0-8ef5794210fe None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:49.677493 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-692fbc9a-d8ec-4d80-af23-fb102455d386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:49.689815 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:49.738021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1d147-aa1b-477f-91f5-d121ac9a0673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:49.738800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1d147-aa1b-477f-91f5-d121ac9a0673 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.739367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1d147-aa1b-477f-91f5-d121ac9a0673 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.739652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1d147-aa1b-477f-91f5-d121ac9a0673 None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.741367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1d147-aa1b-477f-91f5-d121ac9a0673 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:49.744058 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1d147-aa1b-477f-91f5-d121ac9a0673 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:49.745887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 71/137] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:49 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:49.747246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89afa2-2772-40ab-8f6a-be2e2d9c1e94 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.747540 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89afa2-2772-40ab-8f6a-be2e2d9c1e94 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.747819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89afa2-2772-40ab-8f6a-be2e2d9c1e94 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.750090 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 72/138] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:49.752312 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89afa2-2772-40ab-8f6a-be2e2d9c1e94 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:49.826459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 67/139] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:49.828772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3e7a9-8752-4e0c-a9c5-9b24c287c327 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.829460 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3e7a9-8752-4e0c-a9c5-9b24c287c327 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.829968 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3e7a9-8752-4e0c-a9c5-9b24c287c327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.872081 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-34a3e7a9-8752-4e0c-a9c5-9b24c287c327 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:49.873528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-34a3e7a9-8752-4e0c-a9c5-9b24c287c327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:49.892169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89afa2-2772-40ab-8f6a-be2e2d9c1e94 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:49.930629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d8ea9-5069-47e5-b4a9-7ffff9b1d6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:49.931448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d8ea9-5069-47e5-b4a9-7ffff9b1d6fe None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.931805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d8ea9-5069-47e5-b4a9-7ffff9b1d6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.932224 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d8ea9-5069-47e5-b4a9-7ffff9b1d6fe None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69/users/30d09a6b4f0047b4ba7decc0e37bf0cf/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.934409 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d8ea9-5069-47e5-b4a9-7ffff9b1d6fe None admin] RBAC: Authorizing `identity:create_grant(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69, user_id=30d09a6b4f0047b4ba7decc0e37bf0cf, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:49.936128 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d8ea9-5069-47e5-b4a9-7ffff9b1d6fe None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:50.011154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 73/140] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:04:49 2022] PUT /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69/users/30d09a6b4f0047b4ba7decc0e37bf0cf/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:50.020531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 68/141] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:50.500829 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d8ea9-5069-47e5-b4a9-7ffff9b1d6fe None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:50.504762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:50.558769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3e7a9-8752-4e0c-a9c5-9b24c287c327 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:50.741578 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d8ea9-5069-47e5-b4a9-7ffff9b1d6fe None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:50.773007 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:50.803022 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4091b-1be4-42fb-a5e2-fbb1b789a6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:50.805135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4091b-1be4-42fb-a5e2-fbb1b789a6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:50.805135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4091b-1be4-42fb-a5e2-fbb1b789a6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:50.805401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4091b-1be4-42fb-a5e2-fbb1b789a6e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:50.828857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4091b-1be4-42fb-a5e2-fbb1b789a6e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:50.839825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-e9e4091b-1be4-42fb-a5e2-fbb1b789a6e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:50.839825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:50.839825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:50.839825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:50.839825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:50.839825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:50.839825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:50.839825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:50.839825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:50.839825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:50.839825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:50.839825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:50.839825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:50.839825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:50.839825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:50.839825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:50.840863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:50.840863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:50.840863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:50.840863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:50.840863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:50.840863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:50.840863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:50.840863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:50.840863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:50.840863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:50.840863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:50.840863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:50.840863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:50.840863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:50.840863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:50.840863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:50.842378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:50.842378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:50.842378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:50.842378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:50.842378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:50.842378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:50.842378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:50.842378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:50.842378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:50.842378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:50.842378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:50.842378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:04:50.842378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 74/142] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:50.852962 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4091b-1be4-42fb-a5e2-fbb1b789a6e6 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:50.863027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3e7a9-8752-4e0c-a9c5-9b24c287c327 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:50.945004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:50.981794 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb75e4-97b5-457c-910c-9cc7b1f2b7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:50.982831 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb75e4-97b5-457c-910c-9cc7b1f2b7a5 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:50.983357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb75e4-97b5-457c-910c-9cc7b1f2b7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:50.983824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb75e4-97b5-457c-910c-9cc7b1f2b7a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:50.991724 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb75e4-97b5-457c-910c-9cc7b1f2b7a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:50.994551 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb75e4-97b5-457c-910c-9cc7b1f2b7a5 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:50.996918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 69/143] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:51.003055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb75e4-97b5-457c-910c-9cc7b1f2b7a5 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:51.015263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4091b-1be4-42fb-a5e2-fbb1b789a6e6 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:51.059774 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e2c40-068c-4dbc-ad23-e654af2508af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:51.060441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e2c40-068c-4dbc-ad23-e654af2508af None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:51.060719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e2c40-068c-4dbc-ad23-e654af2508af None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:51.060990 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e2c40-068c-4dbc-ad23-e654af2508af None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:51.061694 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e2c40-068c-4dbc-ad23-e654af2508af None admin] RBAC: Adding query filter params (name=member) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:51.061994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e2c40-068c-4dbc-ad23-e654af2508af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:51.066905 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e2c40-068c-4dbc-ad23-e654af2508af None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:51.081486 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 75/144] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:04:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:51.088231 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e2c40-068c-4dbc-ad23-e654af2508af None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:51.289390 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb75e4-97b5-457c-910c-9cc7b1f2b7a5 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:51.349014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e2c40-068c-4dbc-ad23-e654af2508af None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:51.388182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:51.401156 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7c89c-b73f-473f-98db-612182cb5801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:51.402555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7c89c-b73f-473f-98db-612182cb5801 None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:51.402555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7c89c-b73f-473f-98db-612182cb5801 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:51.402555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7c89c-b73f-473f-98db-612182cb5801 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:51.403044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7c89c-b73f-473f-98db-612182cb5801 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:51.404605 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7c89c-b73f-473f-98db-612182cb5801 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:51.404949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ced70d-f39d-457d-8d90-9d30c4f55a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:51.405605 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ced70d-f39d-457d-8d90-9d30c4f55a0f None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:51.405890 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ced70d-f39d-457d-8d90-9d30c4f55a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:51.406168 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ced70d-f39d-457d-8d90-9d30c4f55a0f None admin] PATH_INFO: `/v3/users/7404abca54674994bf5e6ec8f024692d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:51.411567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ced70d-f39d-457d-8d90-9d30c4f55a0f None admin] RBAC: Authorizing `identity:get_user(user_id=7404abca54674994bf5e6ec8f024692d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:51.413715 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ced70d-f39d-457d-8d90-9d30c4f55a0f None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:51.415555 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 76/145] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:51 2022] GET /identity/v3/users/7404abca54674994bf5e6ec8f024692d => generated 275 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:51.423270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ced70d-f39d-457d-8d90-9d30c4f55a0f None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:51.442178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:51.558270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 70/146] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:04:51 2022] POST /identity/v3/users => generated 283 bytes in 555 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:04:51.575537 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ced70d-f39d-457d-8d90-9d30c4f55a0f None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:51.622665 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955be2c-af59-49b4-9465-bfc230af09b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:51.623260 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955be2c-af59-49b4-9465-bfc230af09b2 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:51.623545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955be2c-af59-49b4-9465-bfc230af09b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:51.623825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955be2c-af59-49b4-9465-bfc230af09b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:51.674064 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107c39d-e218-4c15-92e9-19fb492997db None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:51.674837 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107c39d-e218-4c15-92e9-19fb492997db None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:51.674837 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107c39d-e218-4c15-92e9-19fb492997db None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:51.676340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 71/147] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:04:51.684683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84aa37c-66b5-4552-927e-6b8daf107f96 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:51.685054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84aa37c-66b5-4552-927e-6b8daf107f96 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:51.685255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84aa37c-66b5-4552-927e-6b8daf107f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:51.708310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-f84aa37c-66b5-4552-927e-6b8daf107f96 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:51.710993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f84aa37c-66b5-4552-927e-6b8daf107f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:51.754183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:51.770579 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955be2c-af59-49b4-9465-bfc230af09b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:51.772735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955be2c-af59-49b4-9465-bfc230af09b2 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:51.869291 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 72/148] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:51.902028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 77/149] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 19:04:51.908218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84aa37c-66b5-4552-927e-6b8daf107f96 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:52.083948 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84aa37c-66b5-4552-927e-6b8daf107f96 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:52.124641 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:52.134944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc29eba-bd46-4105-ab4e-5f1e646b8f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:52.135593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc29eba-bd46-4105-ab4e-5f1e646b8f14 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.135881 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc29eba-bd46-4105-ab4e-5f1e646b8f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.136158 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc29eba-bd46-4105-ab4e-5f1e646b8f14 None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.146276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc29eba-bd46-4105-ab4e-5f1e646b8f14 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:52.146276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc29eba-bd46-4105-ab4e-5f1e646b8f14 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:52.147651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 73/150] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:51 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:52.154026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955be2c-af59-49b4-9465-bfc230af09b2 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:52.260229 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:52.269389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955be2c-af59-49b4-9465-bfc230af09b2 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:52.301020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc29eba-bd46-4105-ab4e-5f1e646b8f14 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:52.313603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09097505-08f4-4065-a46d-765a99e8c969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:52.314227 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09097505-08f4-4065-a46d-765a99e8c969 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.314519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09097505-08f4-4065-a46d-765a99e8c969 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.314785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09097505-08f4-4065-a46d-765a99e8c969 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.315568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09097505-08f4-4065-a46d-765a99e8c969 None admin] RBAC: Adding query filter params (role.id=357fdcf3417c45319b9d424a5ab4e9ca, scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69, user.id=7404abca54674994bf5e6ec8f024692d) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:52.315946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09097505-08f4-4065-a46d-765a99e8c969 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:52.317421 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09097505-08f4-4065-a46d-765a99e8c969 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:52.326981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 78/151] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:52 2022] GET /identity/v3/role_assignments?user.id=7404abca54674994bf5e6ec8f024692d&scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69&role.id=357fdcf3417c45319b9d424a5ab4e9ca => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:52.512459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc29eba-bd46-4105-ab4e-5f1e646b8f14 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:52.548584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d81bc-7dea-43c0-a37a-9783dec90bc2 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.549263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d81bc-7dea-43c0-a37a-9783dec90bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.549782 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d81bc-7dea-43c0-a37a-9783dec90bc2 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.551663 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 79/152] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:52.557375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be042a-b96b-4149-93c1-0964037b4ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:52.558026 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be042a-b96b-4149-93c1-0964037b4ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.558305 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be042a-b96b-4149-93c1-0964037b4ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.558579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be042a-b96b-4149-93c1-0964037b4ca4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.559638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be644d3-0833-41c8-bb64-faaa28040717 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.560120 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be644d3-0833-41c8-bb64-faaa28040717 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.560583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be644d3-0833-41c8-bb64-faaa28040717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.583840 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be042a-b96b-4149-93c1-0964037b4ca4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:52.589719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-a1be042a-b96b-4149-93c1-0964037b4ca4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:52.589719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:52.589719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:52.589719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:52.589719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:52.589719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:52.589719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:52.589719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:52.589719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:52.589719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:52.589719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:52.589719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:52.589719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:52.589719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:52.589719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:52.589719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:52.590632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:52.590632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:52.590632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:52.590632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:52.590632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:52.590632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:52.590632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:52.590632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:52.590632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:52.590632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:52.590632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:52.590632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:52.590632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:52.590632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:52.590632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:52.590632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:52.591452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:52.591452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:52.591452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:52.591452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:52.591452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:52.591452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:52.591452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:52.591452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:52.591452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:52.591452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:52.591452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:52.591452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:04:52.591452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 74/153] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:52.594969 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-9be644d3-0833-41c8-bb64-faaa28040717 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:52.596295 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-9be644d3-0833-41c8-bb64-faaa28040717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:52.601186 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be042a-b96b-4149-93c1-0964037b4ca4 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:52.702242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:52.716632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 80/154] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:52.722116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e41127-7edc-449c-a3d5-b6393c0e4884 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.722116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e41127-7edc-449c-a3d5-b6393c0e4884 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.722116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e41127-7edc-449c-a3d5-b6393c0e4884 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.722116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 81/155] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:52.722116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07c46c-08bc-4fa6-a3f1-b612932168ad None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.722116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07c46c-08bc-4fa6-a3f1-b612932168ad None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.722694 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07c46c-08bc-4fa6-a3f1-b612932168ad None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.727168 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 82/156] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:52.728433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f9a3e-13d3-4af0-9c99-dacc6f755afd None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.728726 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f9a3e-13d3-4af0-9c99-dacc6f755afd None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.729054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f9a3e-13d3-4af0-9c99-dacc6f755afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.748335 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-008f9a3e-13d3-4af0-9c99-dacc6f755afd None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:52.749530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-008f9a3e-13d3-4af0-9c99-dacc6f755afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:52.761449 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be042a-b96b-4149-93c1-0964037b4ca4 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:52.798846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053009a-a1d0-4f25-8d6e-b036c6a12aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:52.799414 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053009a-a1d0-4f25-8d6e-b036c6a12aeb None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.799756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053009a-a1d0-4f25-8d6e-b036c6a12aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.800034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053009a-a1d0-4f25-8d6e-b036c6a12aeb None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.800707 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053009a-a1d0-4f25-8d6e-b036c6a12aeb None admin] RBAC: Adding query filter params (name=member) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:52.801067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053009a-a1d0-4f25-8d6e-b036c6a12aeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:52.802521 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053009a-a1d0-4f25-8d6e-b036c6a12aeb None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:52.813685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 75/157] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:04:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:52.820928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a41eb-1e8e-42df-b286-90a539e8a60b None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.821220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a41eb-1e8e-42df-b286-90a539e8a60b None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.821499 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a41eb-1e8e-42df-b286-90a539e8a60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.830024 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 83/158] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:52.831322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f9a3e-13d3-4af0-9c99-dacc6f755afd None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:52.841955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-806a41eb-1e8e-42df-b286-90a539e8a60b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:52.843240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-806a41eb-1e8e-42df-b286-90a539e8a60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:52.879029 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:52.917001 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 76/159] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:53.017175 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f9a3e-13d3-4af0-9c99-dacc6f755afd None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:53.045743 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a41eb-1e8e-42df-b286-90a539e8a60b None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:53.061263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd7777-56d8-4472-92e8-ece3d0a545b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:53.062231 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd7777-56d8-4472-92e8-ece3d0a545b4 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:53.062692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd7777-56d8-4472-92e8-ece3d0a545b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:53.063161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd7777-56d8-4472-92e8-ece3d0a545b4 None admin] PATH_INFO: `/v3/users/7404abca54674994bf5e6ec8f024692d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:53.064816 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd7777-56d8-4472-92e8-ece3d0a545b4 None admin] RBAC: Authorizing `identity:get_user(user_id=7404abca54674994bf5e6ec8f024692d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:53.066602 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd7777-56d8-4472-92e8-ece3d0a545b4 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:53.069045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 84/160] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:52 2022] GET /identity/v3/users/7404abca54674994bf5e6ec8f024692d => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:53.076173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd7777-56d8-4472-92e8-ece3d0a545b4 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:53.109080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:53.188848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a41eb-1e8e-42df-b286-90a539e8a60b None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:53.202250 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:53.229015 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd7777-56d8-4472-92e8-ece3d0a545b4 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:53.241091 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81dc58-75b2-48b0-bbff-aa788057fe25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:53.242558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81dc58-75b2-48b0-bbff-aa788057fe25 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:53.243058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81dc58-75b2-48b0-bbff-aa788057fe25 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:53.243915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81dc58-75b2-48b0-bbff-aa788057fe25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:53.258150 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81dc58-75b2-48b0-bbff-aa788057fe25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:53.264609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-cc81dc58-75b2-48b0-bbff-aa788057fe25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:53.264609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:53.264609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:53.264609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:53.264609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:53.264609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:53.264609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:53.264609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:53.264609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:53.264609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:53.264609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:53.264609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:53.264609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:53.264609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:53.264609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:53.264609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:53.269270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:53.269270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:53.269270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:53.269270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:53.269270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:53.269270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:53.269270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:53.269270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:53.269270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:53.269270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:53.269270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:53.269270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:53.269270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:53.269270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:53.269270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:53.269270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:53.270179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:53.270179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:53.270179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:53.270179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:53.270179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:53.270179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:53.270179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:53.270179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:53.270179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:53.270179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:53.270179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:53.270179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:04:53.270179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 77/161] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:04:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:53.273475 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81dc58-75b2-48b0-bbff-aa788057fe25 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:53.301933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216f417-43f9-4db8-8aa9-b99fcb25c869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:53.303350 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216f417-43f9-4db8-8aa9-b99fcb25c869 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:53.303459 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216f417-43f9-4db8-8aa9-b99fcb25c869 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:53.303870 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216f417-43f9-4db8-8aa9-b99fcb25c869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:53.432535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b216f417-43f9-4db8-8aa9-b99fcb25c869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:53.433996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b216f417-43f9-4db8-8aa9-b99fcb25c869 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:53.463105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81dc58-75b2-48b0-bbff-aa788057fe25 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:53.472045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:53.526990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ace47-a88b-4446-88df-4c77ccecde11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:53.527670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ace47-a88b-4446-88df-4c77ccecde11 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:53.527954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ace47-a88b-4446-88df-4c77ccecde11 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:53.528224 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ace47-a88b-4446-88df-4c77ccecde11 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:53.528894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ace47-a88b-4446-88df-4c77ccecde11 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:53.529197 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ace47-a88b-4446-88df-4c77ccecde11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:53.530601 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ace47-a88b-4446-88df-4c77ccecde11 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:53.547836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 78/162] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:53.548724 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ace47-a88b-4446-88df-4c77ccecde11 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:53.598078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:53.631110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 85/163] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:53.632461 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216f417-43f9-4db8-8aa9-b99fcb25c869 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:53.745425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ace47-a88b-4446-88df-4c77ccecde11 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:53.816218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57d3b3-ccd8-4890-b4da-4ae5b223916c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:53.816866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57d3b3-ccd8-4890-b4da-4ae5b223916c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:53.817148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57d3b3-ccd8-4890-b4da-4ae5b223916c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:53.817418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57d3b3-ccd8-4890-b4da-4ae5b223916c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:53.820511 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216f417-43f9-4db8-8aa9-b99fcb25c869 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:53.840767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd57d3b3-ccd8-4890-b4da-4ae5b223916c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:53.846114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-bd57d3b3-ccd8-4890-b4da-4ae5b223916c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:53.846114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:53.846114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:53.846114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:53.846114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:53.846114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:53.846114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:53.846114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:53.846114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:53.846114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:53.846114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:53.846114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:53.846114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:53.846114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:53.846114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:53.846114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:53.847042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:53.847042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:53.847042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:53.847042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:53.847042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:53.847042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:53.847042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:53.847042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:53.847042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:53.847042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:53.847042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:53.847042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:53.847042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:53.847042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:53.847042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:53.847042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:53.848789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:53.848789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:53.848789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:53.848789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:53.848789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:53.848789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:53.848789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:53.848789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:53.848789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:53.848789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:53.848789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:53.848789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:04:53.848789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 79/164] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:53.851073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57d3b3-ccd8-4890-b4da-4ae5b223916c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:53.895639 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0cc96-f747-49a1-b345-285a362efb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:53.896285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0cc96-f747-49a1-b345-285a362efb9c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:53.897195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0cc96-f747-49a1-b345-285a362efb9c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:53.897692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0cc96-f747-49a1-b345-285a362efb9c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:53.899289 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:53.919254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0cc96-f747-49a1-b345-285a362efb9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:53.924218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-7cd0cc96-f747-49a1-b345-285a362efb9c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:04:53.924218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:53.924218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:53.924218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:53.924218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:53.924218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:53.924218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:53.924218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:53.924218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:53.924218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:53.924218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:53.924218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:53.924218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:53.924218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:53.924218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:53.924218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:53.925454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:53.925454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:53.925454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:53.925454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:53.925454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:53.925454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:53.925454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:53.925454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:53.925454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:53.925454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:53.925454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:53.925454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:53.925454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:53.925454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:53.925454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:53.925454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:53.926253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:53.926253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:53.926253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:53.926253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:53.926253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:53.926253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:53.926253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:53.926253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:53.926253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:53.926253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:53.926253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:04:53.926253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:04:53.926253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 86/165] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:53.926253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0cc96-f747-49a1-b345-285a362efb9c None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:53.986087 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:54.053060 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57d3b3-ccd8-4890-b4da-4ae5b223916c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:54.090826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0cc96-f747-49a1-b345-285a362efb9c None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:54.094408 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc194b95-0a84-4399-ae44-6c7e8de59743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:54.095072 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc194b95-0a84-4399-ae44-6c7e8de59743 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.095355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc194b95-0a84-4399-ae44-6c7e8de59743 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.095631 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc194b95-0a84-4399-ae44-6c7e8de59743 None admin] PATH_INFO: `/v3/users/7404abca54674994bf5e6ec8f024692d` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.102295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc194b95-0a84-4399-ae44-6c7e8de59743 None admin] RBAC: Authorizing `identity:get_user(user_id=7404abca54674994bf5e6ec8f024692d)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:54.103797 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc194b95-0a84-4399-ae44-6c7e8de59743 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:54.105510 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 80/166] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:53 2022] GET /identity/v3/users/7404abca54674994bf5e6ec8f024692d => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:54.106287 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc194b95-0a84-4399-ae44-6c7e8de59743 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:54.188579 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01419f0-10ed-435f-ac92-f645e23f03b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:54.189226 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01419f0-10ed-435f-ac92-f645e23f03b3 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.189509 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01419f0-10ed-435f-ac92-f645e23f03b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.189886 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01419f0-10ed-435f-ac92-f645e23f03b3 None admin] PATH_INFO: `/v3/projects/96a275e837304a4f81ef4efa38975cca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.193982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01419f0-10ed-435f-ac92-f645e23f03b3 None admin] RBAC: Authorizing `identity:get_project(project_id=96a275e837304a4f81ef4efa38975cca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:54.195741 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01419f0-10ed-435f-ac92-f645e23f03b3 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:54.197496 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 87/167] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:53 2022] GET /identity/v3/projects/96a275e837304a4f81ef4efa38975cca => generated 312 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:54.198279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01419f0-10ed-435f-ac92-f645e23f03b3 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:54.214230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:54.220325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc194b95-0a84-4399-ae44-6c7e8de59743 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:54.259487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b5d03-43f7-4b81-ba8f-21f049bb1374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:54.260204 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b5d03-43f7-4b81-ba8f-21f049bb1374 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.260503 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b5d03-43f7-4b81-ba8f-21f049bb1374 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.260866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b5d03-43f7-4b81-ba8f-21f049bb1374 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.261545 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b5d03-43f7-4b81-ba8f-21f049bb1374 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:54.261844 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b5d03-43f7-4b81-ba8f-21f049bb1374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:54.263277 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b5d03-43f7-4b81-ba8f-21f049bb1374 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:54.277967 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 81/168] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:54.278709 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b5d03-43f7-4b81-ba8f-21f049bb1374 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:54.392958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01419f0-10ed-435f-ac92-f645e23f03b3 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:54.397560 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b5d03-43f7-4b81-ba8f-21f049bb1374 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:54.417515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:54.437336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544d9ce-fd31-4e2a-8760-6d27b79a9920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:54.437921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544d9ce-fd31-4e2a-8760-6d27b79a9920 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.438201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544d9ce-fd31-4e2a-8760-6d27b79a9920 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.438475 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544d9ce-fd31-4e2a-8760-6d27b79a9920 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.460764 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de8cea-943d-4ead-9f30-221b6eb64107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:54.461389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de8cea-943d-4ead-9f30-221b6eb64107 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.461674 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de8cea-943d-4ead-9f30-221b6eb64107 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.461998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de8cea-943d-4ead-9f30-221b6eb64107 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.462687 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de8cea-943d-4ead-9f30-221b6eb64107 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:54.463060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de8cea-943d-4ead-9f30-221b6eb64107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:54.464451 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de8cea-943d-4ead-9f30-221b6eb64107 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:54.482791 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 88/169] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:04:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:54.483726 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de8cea-943d-4ead-9f30-221b6eb64107 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:54.515320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:54.525593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544d9ce-fd31-4e2a-8760-6d27b79a9920 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:54.527469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544d9ce-fd31-4e2a-8760-6d27b79a9920 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:54.606318 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de8cea-943d-4ead-9f30-221b6eb64107 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:54.641877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a72c8-f843-40e3-bfe5-ff52dd1daa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:54.642492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a72c8-f843-40e3-bfe5-ff52dd1daa20 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.643039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a72c8-f843-40e3-bfe5-ff52dd1daa20 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.643263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a72c8-f843-40e3-bfe5-ff52dd1daa20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.643998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a72c8-f843-40e3-bfe5-ff52dd1daa20 None admin] RBAC: Adding query filter params (role.id=357fdcf3417c45319b9d424a5ab4e9ca, scope.project.id=96a275e837304a4f81ef4efa38975cca, user.id=7404abca54674994bf5e6ec8f024692d) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:54.644382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a72c8-f843-40e3-bfe5-ff52dd1daa20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:54.645887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a72c8-f843-40e3-bfe5-ff52dd1daa20 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:54.652820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 89/170] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:54 2022] GET /identity/v3/role_assignments?user.id=7404abca54674994bf5e6ec8f024692d&scope.project.id=96a275e837304a4f81ef4efa38975cca&role.id=357fdcf3417c45319b9d424a5ab4e9ca => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:54.653693 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a72c8-f843-40e3-bfe5-ff52dd1daa20 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:54.703392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:54.776584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a72c8-f843-40e3-bfe5-ff52dd1daa20 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:54.797731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 82/171] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:54.804484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544d9ce-fd31-4e2a-8760-6d27b79a9920 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:54.852758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85088579-07bb-4d94-a8ec-04965a8c946b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:54.853419 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088579-07bb-4d94-a8ec-04965a8c946b None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.853702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088579-07bb-4d94-a8ec-04965a8c946b None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.853977 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088579-07bb-4d94-a8ec-04965a8c946b None admin] PATH_INFO: `/v3/users/30d09a6b4f0047b4ba7decc0e37bf0cf` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.859780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85088579-07bb-4d94-a8ec-04965a8c946b None admin] RBAC: Authorizing `identity:get_user(user_id=30d09a6b4f0047b4ba7decc0e37bf0cf)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:54.861266 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85088579-07bb-4d94-a8ec-04965a8c946b None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:54.863039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 90/172] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:54 2022] GET /identity/v3/users/30d09a6b4f0047b4ba7decc0e37bf0cf => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:54.864631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4683bd0-b2b9-4910-8997-1c8555541aeb None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.864919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4683bd0-b2b9-4910-8997-1c8555541aeb None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.865231 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4683bd0-b2b9-4910-8997-1c8555541aeb None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.866554 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 91/173] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:54.870592 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4683bd0-b2b9-4910-8997-1c8555541aeb None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:54.898264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:54.956962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544d9ce-fd31-4e2a-8760-6d27b79a9920 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:55.010433 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1333249-7fbc-41e6-a83f-015656c4a621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:55.011138 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1333249-7fbc-41e6-a83f-015656c4a621 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.011415 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1333249-7fbc-41e6-a83f-015656c4a621 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.011685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1333249-7fbc-41e6-a83f-015656c4a621 None admin] PATH_INFO: `/v3/users/3932ff5e04dd4aadbc9ba0548512f2be` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.012899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1333249-7fbc-41e6-a83f-015656c4a621 None admin] RBAC: Authorizing `identity:get_user(user_id=3932ff5e04dd4aadbc9ba0548512f2be)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:55.014407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1333249-7fbc-41e6-a83f-015656c4a621 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:55.016376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 83/174] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:54 2022] GET /identity/v3/users/3932ff5e04dd4aadbc9ba0548512f2be => generated 289 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:55.017294 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1333249-7fbc-41e6-a83f-015656c4a621 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:55.036087 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4683bd0-b2b9-4910-8997-1c8555541aeb None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:55.045234 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:55.078802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c2628-16d2-4ec5-a9a2-179e105d7bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:55.079476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c2628-16d2-4ec5-a9a2-179e105d7bac None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.079758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c2628-16d2-4ec5-a9a2-179e105d7bac None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.080036 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c2628-16d2-4ec5-a9a2-179e105d7bac None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.081272 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c2628-16d2-4ec5-a9a2-179e105d7bac None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:55.082833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c2628-16d2-4ec5-a9a2-179e105d7bac None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:55.085024 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 92/175] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:54 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:55.086366 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b025f7-d050-4d5c-9ffb-3dbcbb959c7a None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.086651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b025f7-d050-4d5c-9ffb-3dbcbb959c7a None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.088841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b025f7-d050-4d5c-9ffb-3dbcbb959c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.111269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-20b025f7-d050-4d5c-9ffb-3dbcbb959c7a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:55.112560 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-20b025f7-d050-4d5c-9ffb-3dbcbb959c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:55.157131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1333249-7fbc-41e6-a83f-015656c4a621 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:55.187110 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:55.196884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847814ec-c4ec-41c0-b500-4727f17444d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:55.197622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847814ec-c4ec-41c0-b500-4727f17444d1 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.197903 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847814ec-c4ec-41c0-b500-4727f17444d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.198229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847814ec-c4ec-41c0-b500-4727f17444d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.204717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 93/176] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:55.210864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b025f7-d050-4d5c-9ffb-3dbcbb959c7a None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:55.344440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b025f7-d050-4d5c-9ffb-3dbcbb959c7a None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:55.347084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:55.387055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bb37b-15fe-4505-a31f-66c843821ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:55.387846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bb37b-15fe-4505-a31f-66c843821ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.388246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bb37b-15fe-4505-a31f-66c843821ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.388518 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bb37b-15fe-4505-a31f-66c843821ca6 None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.389848 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bb37b-15fe-4505-a31f-66c843821ca6 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:55.391362 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bb37b-15fe-4505-a31f-66c843821ca6 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:55.393223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 94/177] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:55 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:55.394338 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bb37b-15fe-4505-a31f-66c843821ca6 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:55.415996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847814ec-c4ec-41c0-b500-4727f17444d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:55.417416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847814ec-c4ec-41c0-b500-4727f17444d1 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:55.513577 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bb37b-15fe-4505-a31f-66c843821ca6 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:55.561029 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fd92a-9d93-4725-98b5-960f5ce5daa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:55.561912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fd92a-9d93-4725-98b5-960f5ce5daa4 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.562188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fd92a-9d93-4725-98b5-960f5ce5daa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.562461 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fd92a-9d93-4725-98b5-960f5ce5daa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.563642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fd92a-9d93-4725-98b5-960f5ce5daa4 None admin] RBAC: Adding query filter params (role.id=e14f37f4a3a24f8da0f757aabb1f39be, scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69, user.id=7404abca54674994bf5e6ec8f024692d) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:55.564823 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fd92a-9d93-4725-98b5-960f5ce5daa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:55.566272 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fd92a-9d93-4725-98b5-960f5ce5daa4 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:55.574488 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 95/178] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:55 2022] GET /identity/v3/role_assignments?user.id=7404abca54674994bf5e6ec8f024692d&scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69&role.id=e14f37f4a3a24f8da0f757aabb1f39be => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:55.575773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fd92a-9d93-4725-98b5-960f5ce5daa4 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:55.601966 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 84/179] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:55.603487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059462c-3cb3-44b0-80b8-b3ddea793614 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.603847 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059462c-3cb3-44b0-80b8-b3ddea793614 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.604197 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059462c-3cb3-44b0-80b8-b3ddea793614 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.605621 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 85/180] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:55.606395 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059462c-3cb3-44b0-80b8-b3ddea793614 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:55.630489 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:55.640313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:55.712581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fd92a-9d93-4725-98b5-960f5ce5daa4 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:55.725113 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059462c-3cb3-44b0-80b8-b3ddea793614 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:55.758085 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330059f-5b9d-4ba2-93b5-40f936313b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:55.758767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330059f-5b9d-4ba2-93b5-40f936313b21 None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.759880 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330059f-5b9d-4ba2-93b5-40f936313b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.760166 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330059f-5b9d-4ba2-93b5-40f936313b21 None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69/users/3932ff5e04dd4aadbc9ba0548512f2be/roles/6c1ffb941f1646d2bf707005c7796790` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.762106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a330059f-5b9d-4ba2-93b5-40f936313b21 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69, user_id=3932ff5e04dd4aadbc9ba0548512f2be, role_id=6c1ffb941f1646d2bf707005c7796790)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:55.764345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a330059f-5b9d-4ba2-93b5-40f936313b21 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:55.769824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96949349-b78d-44d2-ae8a-5efb9667d44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:55.770557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96949349-b78d-44d2-ae8a-5efb9667d44b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.770936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96949349-b78d-44d2-ae8a-5efb9667d44b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.771286 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96949349-b78d-44d2-ae8a-5efb9667d44b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.784753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96949349-b78d-44d2-ae8a-5efb9667d44b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:55.789215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-96949349-b78d-44d2-ae8a-5efb9667d44b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:55.789215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:55.789215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:55.789215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:55.789215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:55.789215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:55.789215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:55.789215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:55.789215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:55.789215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:55.789215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:55.789215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:55.789215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:55.789215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:55.789215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:55.789215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:55.790200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:55.790200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:55.790200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:55.790200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:55.790200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:55.790200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:55.790200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:55.790200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:55.790200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:55.790200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:55.790200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:55.790200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:55.790200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:55.790200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:55.790200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:55.790200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:55.791057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:55.791057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:55.791057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:55.791057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:55.791057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:55.791057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:55.791057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:55.791057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:55.791057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:55.791057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:55.791057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:55.791057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:04:55.791057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 86/181] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:55.791739 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e394fc8-294f-477c-be74-00bf176fa83c None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.791805 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e394fc8-294f-477c-be74-00bf176fa83c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.792072 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e394fc8-294f-477c-be74-00bf176fa83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.810686 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-8e394fc8-294f-477c-be74-00bf176fa83c None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:55.811976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8e394fc8-294f-477c-be74-00bf176fa83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:55.821936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 96/182] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:04:55 2022] PUT /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69/users/3932ff5e04dd4aadbc9ba0548512f2be/roles/6c1ffb941f1646d2bf707005c7796790 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:55.822748 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330059f-5b9d-4ba2-93b5-40f936313b21 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:55.897222 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 87/183] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:55.898625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa77862-4ce3-4dfc-9c9e-cc140a383b46 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.899223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa77862-4ce3-4dfc-9c9e-cc140a383b46 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.899458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa77862-4ce3-4dfc-9c9e-cc140a383b46 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.906602 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 88/184] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:55.907501 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa77862-4ce3-4dfc-9c9e-cc140a383b46 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:55.929069 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:55.955749 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:55.963777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330059f-5b9d-4ba2-93b5-40f936313b21 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:56.008065 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6f73e-ade5-487a-b860-ea3e7f277875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:56.009242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6f73e-ade5-487a-b860-ea3e7f277875 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.009698 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6f73e-ade5-487a-b860-ea3e7f277875 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.010165 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6f73e-ade5-487a-b860-ea3e7f277875 None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.011474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6f73e-ade5-487a-b860-ea3e7f277875 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:56.013309 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6f73e-ade5-487a-b860-ea3e7f277875 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:56.015099 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 97/185] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:55 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:56.016818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bc9a3-4f96-412f-bf0c-7097dcdd2f74 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.017282 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bc9a3-4f96-412f-bf0c-7097dcdd2f74 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.017695 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bc9a3-4f96-412f-bf0c-7097dcdd2f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.035154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa77862-4ce3-4dfc-9c9e-cc140a383b46 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:56.039353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-922bc9a3-4f96-412f-bf0c-7097dcdd2f74 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:56.040688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-922bc9a3-4f96-412f-bf0c-7097dcdd2f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:56.077165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094889a-7678-4f0c-acb7-fe1202bdfa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:56.077952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094889a-7678-4f0c-acb7-fe1202bdfa6b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.078350 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094889a-7678-4f0c-acb7-fe1202bdfa6b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.078776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094889a-7678-4f0c-acb7-fe1202bdfa6b None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.079591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094889a-7678-4f0c-acb7-fe1202bdfa6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:56.080029 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094889a-7678-4f0c-acb7-fe1202bdfa6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:56.081985 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094889a-7678-4f0c-acb7-fe1202bdfa6b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:56.092943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 89/186] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:56.097407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094889a-7678-4f0c-acb7-fe1202bdfa6b None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:56.123662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 98/187] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:56.125064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bc9a3-4f96-412f-bf0c-7097dcdd2f74 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:56.200219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:56.221436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094889a-7678-4f0c-acb7-fe1202bdfa6b None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:56.222624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:56.230571 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bc9a3-4f96-412f-bf0c-7097dcdd2f74 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:56.260505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e02435-20ce-4b87-8a0a-2f2e5bcd3e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:56.261150 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e02435-20ce-4b87-8a0a-2f2e5bcd3e06 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.261434 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e02435-20ce-4b87-8a0a-2f2e5bcd3e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.261706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e02435-20ce-4b87-8a0a-2f2e5bcd3e06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.262630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e02435-20ce-4b87-8a0a-2f2e5bcd3e06 None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69, user.id=30d09a6b4f0047b4ba7decc0e37bf0cf) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:56.262747 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e02435-20ce-4b87-8a0a-2f2e5bcd3e06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:56.264216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e02435-20ce-4b87-8a0a-2f2e5bcd3e06 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:56.270397 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 90/188] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:56 2022] GET /identity/v3/role_assignments?user.id=30d09a6b4f0047b4ba7decc0e37bf0cf&scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:56.271479 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e02435-20ce-4b87-8a0a-2f2e5bcd3e06 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:56.274963 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2ec5c-fc8b-4bb6-99ba-7bff7fcc62cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:56.275974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2ec5c-fc8b-4bb6-99ba-7bff7fcc62cb None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.276453 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2ec5c-fc8b-4bb6-99ba-7bff7fcc62cb None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.276930 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2ec5c-fc8b-4bb6-99ba-7bff7fcc62cb None admin] PATH_INFO: `/v3/users/50d15c2581a04b59995fa2eb734bacc4` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.295939 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f2ec5c-fc8b-4bb6-99ba-7bff7fcc62cb None admin] RBAC: Authorizing `identity:get_user(user_id=50d15c2581a04b59995fa2eb734bacc4)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:56.297960 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f2ec5c-fc8b-4bb6-99ba-7bff7fcc62cb None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:56.300204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 99/189] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:56 2022] GET /identity/v3/users/50d15c2581a04b59995fa2eb734bacc4 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:56.307716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2ec5c-fc8b-4bb6-99ba-7bff7fcc62cb None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:56.390100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e02435-20ce-4b87-8a0a-2f2e5bcd3e06 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:56.453933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2ec5c-fc8b-4bb6-99ba-7bff7fcc62cb None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:56.464553 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2deb4b4-0554-424c-a42a-4e4028fd2e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:56.465209 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deb4b4-0554-424c-a42a-4e4028fd2e3c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.465608 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deb4b4-0554-424c-a42a-4e4028fd2e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.465887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deb4b4-0554-424c-a42a-4e4028fd2e3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.467103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2deb4b4-0554-424c-a42a-4e4028fd2e3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:56.468577 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2deb4b4-0554-424c-a42a-4e4028fd2e3c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:56.470666 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 91/190] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:56.476821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2deb4b4-0554-424c-a42a-4e4028fd2e3c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:56.494604 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:56.501854 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187170af-5be9-4730-a3a6-febde3f7f726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:56.502832 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187170af-5be9-4730-a3a6-febde3f7f726 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.503215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187170af-5be9-4730-a3a6-febde3f7f726 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.503497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187170af-5be9-4730-a3a6-febde3f7f726 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.534004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:56.593365 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187170af-5be9-4730-a3a6-febde3f7f726 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:56.594830 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187170af-5be9-4730-a3a6-febde3f7f726 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:56.607244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2deb4b4-0554-424c-a42a-4e4028fd2e3c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:56.645529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687fbe8-8941-4f92-85fc-948e267a94c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:56.646147 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687fbe8-8941-4f92-85fc-948e267a94c6 None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.647038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687fbe8-8941-4f92-85fc-948e267a94c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.647286 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687fbe8-8941-4f92-85fc-948e267a94c6 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.648034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687fbe8-8941-4f92-85fc-948e267a94c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:56.649480 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687fbe8-8941-4f92-85fc-948e267a94c6 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:56.698855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 100/191] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:56.699754 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187170af-5be9-4730-a3a6-febde3f7f726 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:56.719269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 92/192] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:04:56 2022] POST /identity/v3/users => generated 297 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:04:56.720281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687fbe8-8941-4f92-85fc-948e267a94c6 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:56.778510 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:56.830470 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:56.841316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187170af-5be9-4730-a3a6-febde3f7f726 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:56.874796 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687fbe8-8941-4f92-85fc-948e267a94c6 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:56.885356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10f80e-8bc1-449c-ae7b-6fd10a89238b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:56.886467 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10f80e-8bc1-449c-ae7b-6fd10a89238b None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.886751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10f80e-8bc1-449c-ae7b-6fd10a89238b None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.887076 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10f80e-8bc1-449c-ae7b-6fd10a89238b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.903676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10f80e-8bc1-449c-ae7b-6fd10a89238b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:56.908624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-fe10f80e-8bc1-449c-ae7b-6fd10a89238b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:56.908624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:56.908624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:56.908624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:56.908624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:56.908624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:56.908624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:56.908624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:56.908624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:56.908624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:56.908624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:56.908624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:56.908624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:56.908624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:56.908624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:56.908624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:56.909551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:56.909551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:56.909551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:56.909551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:56.909551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:56.909551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:56.909551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:56.909551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:56.909551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:56.909551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:56.909551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:56.909551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:56.909551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:56.909551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:56.909551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:56.909551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:56.910358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:56.910358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:56.910358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:56.910358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:56.910358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:56.910358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:56.910358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:56.910358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:56.910358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:56.910358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:56.910358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:56.910358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:04:56.910358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 101/193] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:04:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:56.915876 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10f80e-8bc1-449c-ae7b-6fd10a89238b None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:56.924051 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e782d-36f2-4b83-a47f-e8382b02f9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:56.924792 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e782d-36f2-4b83-a47f-e8382b02f9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.925072 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e782d-36f2-4b83-a47f-e8382b02f9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.925353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e782d-36f2-4b83-a47f-e8382b02f9a4 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.927024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e782d-36f2-4b83-a47f-e8382b02f9a4 None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:56.929401 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e782d-36f2-4b83-a47f-e8382b02f9a4 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:56.932494 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 93/194] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:56 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:56.938851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e782d-36f2-4b83-a47f-e8382b02f9a4 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:57.073676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10f80e-8bc1-449c-ae7b-6fd10a89238b None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:57.079520 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:57.104732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e782d-36f2-4b83-a47f-e8382b02f9a4 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:57.114036 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fd382-68a6-4235-9d35-25112140235d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:57.114658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fd382-68a6-4235-9d35-25112140235d None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:57.114996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fd382-68a6-4235-9d35-25112140235d None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:57.115369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fd382-68a6-4235-9d35-25112140235d None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:57.116208 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fd382-68a6-4235-9d35-25112140235d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:57.116513 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fd382-68a6-4235-9d35-25112140235d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:57.117949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fd382-68a6-4235-9d35-25112140235d None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:57.127522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 102/195] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:57.135983 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fd382-68a6-4235-9d35-25112140235d None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:57.147826 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677e9e5-8692-4868-9e6d-2e7e0ac39c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:57.148062 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677e9e5-8692-4868-9e6d-2e7e0ac39c7b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:57.148424 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677e9e5-8692-4868-9e6d-2e7e0ac39c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:57.148708 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677e9e5-8692-4868-9e6d-2e7e0ac39c7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:57.149480 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677e9e5-8692-4868-9e6d-2e7e0ac39c7b None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d, user.id=50d15c2581a04b59995fa2eb734bacc4) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:57.149904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677e9e5-8692-4868-9e6d-2e7e0ac39c7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:57.151797 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677e9e5-8692-4868-9e6d-2e7e0ac39c7b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:57.154284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:57.170512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 94/196] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:56 2022] GET /identity/v3/role_assignments?user.id=50d15c2581a04b59995fa2eb734bacc4&scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:57.269424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fd382-68a6-4235-9d35-25112140235d None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:57.322556 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd39b99-b1a6-49c9-bff8-4e72e397b1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:57.323325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd39b99-b1a6-49c9-bff8-4e72e397b1aa None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:57.323837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd39b99-b1a6-49c9-bff8-4e72e397b1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:57.324257 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd39b99-b1a6-49c9-bff8-4e72e397b1aa None admin] PATH_INFO: `/v3/users/50d15c2581a04b59995fa2eb734bacc4` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:57.331857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd39b99-b1a6-49c9-bff8-4e72e397b1aa None admin] RBAC: Authorizing `identity:get_user(user_id=50d15c2581a04b59995fa2eb734bacc4)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:57.333439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd39b99-b1a6-49c9-bff8-4e72e397b1aa None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:57.336401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 103/197] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:57 2022] GET /identity/v3/users/50d15c2581a04b59995fa2eb734bacc4 => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:57.343099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677e9e5-8692-4868-9e6d-2e7e0ac39c7b None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:57.489116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677e9e5-8692-4868-9e6d-2e7e0ac39c7b None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:57.538722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb95f51-9916-4a4c-bcd9-a224dc648116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:57.539385 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb95f51-9916-4a4c-bcd9-a224dc648116 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:57.539671 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb95f51-9916-4a4c-bcd9-a224dc648116 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:57.539950 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb95f51-9916-4a4c-bcd9-a224dc648116 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:57.627761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:57.631870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb95f51-9916-4a4c-bcd9-a224dc648116 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:57.634251 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb95f51-9916-4a4c-bcd9-a224dc648116 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:57.785287 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 95/198] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:57.792453 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd39b99-b1a6-49c9-bff8-4e72e397b1aa None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:57.871394 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:57.948550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd39b99-b1a6-49c9-bff8-4e72e397b1aa None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:57.992591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e98ead-be14-452c-aa6b-bb5dc89eddec None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:57.993126 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e98ead-be14-452c-aa6b-bb5dc89eddec None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:57.993126 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e98ead-be14-452c-aa6b-bb5dc89eddec None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:57.994698 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 96/199] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:04:58.000060 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009d30a-6c3b-42d0-af97-9a21728591ff None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:58.000336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009d30a-6c3b-42d0-af97-9a21728591ff None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:58.000614 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009d30a-6c3b-42d0-af97-9a21728591ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:58.005232 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df248c-198a-489c-a483-36965dff4fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:58.006694 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df248c-198a-489c-a483-36965dff4fef None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:58.007221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df248c-198a-489c-a483-36965dff4fef None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:58.007712 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df248c-198a-489c-a483-36965dff4fef None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:58.009165 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df248c-198a-489c-a483-36965dff4fef None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:58.010617 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df248c-198a-489c-a483-36965dff4fef None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:58.013331 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 104/200] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:57 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:58.022782 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df248c-198a-489c-a483-36965dff4fef None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:58.033479 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d009d30a-6c3b-42d0-af97-9a21728591ff None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:58.046911 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d009d30a-6c3b-42d0-af97-9a21728591ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:58.163011 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 97/201] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:58.206580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df248c-198a-489c-a483-36965dff4fef None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:58.226369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:58.250803 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a325dad-c44c-4b12-b42a-8a68a60fc3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:58.251743 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a325dad-c44c-4b12-b42a-8a68a60fc3be None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:58.252214 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a325dad-c44c-4b12-b42a-8a68a60fc3be None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:58.252661 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a325dad-c44c-4b12-b42a-8a68a60fc3be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:58.253552 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a325dad-c44c-4b12-b42a-8a68a60fc3be None admin] RBAC: Adding query filter params (role.id=e14f37f4a3a24f8da0f757aabb1f39be, scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d, user.id=50d15c2581a04b59995fa2eb734bacc4) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:58.254099 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a325dad-c44c-4b12-b42a-8a68a60fc3be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:58.256408 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a325dad-c44c-4b12-b42a-8a68a60fc3be None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:58.262858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 105/202] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:58 2022] GET /identity/v3/role_assignments?user.id=50d15c2581a04b59995fa2eb734bacc4&scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d&role.id=e14f37f4a3a24f8da0f757aabb1f39be => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:58.497863 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009d30a-6c3b-42d0-af97-9a21728591ff None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:58.528877 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:58.605708 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009d30a-6c3b-42d0-af97-9a21728591ff None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:58.643557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41322787-5b72-4a5c-94ef-d06c6cfdb961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:58.644621 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41322787-5b72-4a5c-94ef-d06c6cfdb961 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:58.645107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41322787-5b72-4a5c-94ef-d06c6cfdb961 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:58.645573 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41322787-5b72-4a5c-94ef-d06c6cfdb961 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:58.657558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41322787-5b72-4a5c-94ef-d06c6cfdb961 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:58.663326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-41322787-5b72-4a5c-94ef-d06c6cfdb961 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:04:58.663326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:58.663326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:58.663326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:58.663326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:58.663326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:58.663326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:58.663326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:58.663326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:58.663326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:58.663326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:58.663326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:58.663326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:58.663326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:58.663326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:58.663326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:58.664338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:58.664338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:58.664338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:58.664338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:58.664338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:58.664338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:58.664338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:58.664338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:58.664338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:58.664338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:58.664338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:58.664338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:58.664338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:58.664338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:58.664338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:58.664338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:58.665203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:58.665203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:58.665203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:58.665203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:58.665203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:58.665203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:58.665203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:58.665203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:58.665203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:58.665203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:58.665203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:04:58.665203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:04:58.667507 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 98/203] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:58.676007 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a325dad-c44c-4b12-b42a-8a68a60fc3be None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:58.809620 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a325dad-c44c-4b12-b42a-8a68a60fc3be None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:58.884948 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b068f-c48a-4f37-bb41-997a2c84ace2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:58.885573 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b068f-c48a-4f37-bb41-997a2c84ace2 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:58.885858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b068f-c48a-4f37-bb41-997a2c84ace2 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:58.886135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b068f-c48a-4f37-bb41-997a2c84ace2 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:58.886812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b068f-c48a-4f37-bb41-997a2c84ace2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:58.887143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b068f-c48a-4f37-bb41-997a2c84ace2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:58.888530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b068f-c48a-4f37-bb41-997a2c84ace2 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:58.902034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 106/204] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:04:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:58.907638 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41322787-5b72-4a5c-94ef-d06c6cfdb961 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:59.031314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:59.040889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41322787-5b72-4a5c-94ef-d06c6cfdb961 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:59.076862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af683eb4-ae1f-401d-80f2-dcf09e6a7f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:59.077936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af683eb4-ae1f-401d-80f2-dcf09e6a7f2d None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:59.078469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af683eb4-ae1f-401d-80f2-dcf09e6a7f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:59.078988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af683eb4-ae1f-401d-80f2-dcf09e6a7f2d None admin] PATH_INFO: `/v3/users/3932ff5e04dd4aadbc9ba0548512f2be` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:59.080340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af683eb4-ae1f-401d-80f2-dcf09e6a7f2d None admin] RBAC: Authorizing `identity:get_user(user_id=3932ff5e04dd4aadbc9ba0548512f2be)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:59.082096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af683eb4-ae1f-401d-80f2-dcf09e6a7f2d None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:59.089183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 99/205] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:58 2022] GET /identity/v3/users/3932ff5e04dd4aadbc9ba0548512f2be => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:59.091071 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b068f-c48a-4f37-bb41-997a2c84ace2 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:59.136481 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:59.206426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b068f-c48a-4f37-bb41-997a2c84ace2 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:59.243104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58527b7-266f-422c-b739-08b78d05f2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:59.244314 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58527b7-266f-422c-b739-08b78d05f2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:59.244811 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58527b7-266f-422c-b739-08b78d05f2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:59.245339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58527b7-266f-422c-b739-08b78d05f2d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:59.398899 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58527b7-266f-422c-b739-08b78d05f2d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:59.431640 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58527b7-266f-422c-b739-08b78d05f2d0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:59.436872 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:59.533575 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 107/206] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 19:04:59.539593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af683eb4-ae1f-401d-80f2-dcf09e6a7f2d None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:59.657482 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b37d61-e424-4102-afe6-8cf0c9806332 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:59.657821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b37d61-e424-4102-afe6-8cf0c9806332 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:59.658236 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b37d61-e424-4102-afe6-8cf0c9806332 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:59.659884 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 108/207] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:59.667106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194ccc7-3664-49aa-9ef4-075e84f2f9aa None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:59.667397 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194ccc7-3664-49aa-9ef4-075e84f2f9aa None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:59.667673 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194ccc7-3664-49aa-9ef4-075e84f2f9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:59.689342 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-8194ccc7-3664-49aa-9ef4-075e84f2f9aa None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:59.690541 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8194ccc7-3664-49aa-9ef4-075e84f2f9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:59.720342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af683eb4-ae1f-401d-80f2-dcf09e6a7f2d None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:59.769283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9c6bd-8bce-4cf2-ba5f-580d163fc3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:59.770983 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9c6bd-8bce-4cf2-ba5f-580d163fc3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:59.771937 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9c6bd-8bce-4cf2-ba5f-580d163fc3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:59.772401 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9c6bd-8bce-4cf2-ba5f-580d163fc3e4 None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:59.774248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9c6bd-8bce-4cf2-ba5f-580d163fc3e4 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:59.776140 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9c6bd-8bce-4cf2-ba5f-580d163fc3e4 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:59.778354 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 100/208] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:59 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:59.779733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545c6f5-4a43-4a38-9e54-28559eb4a1a4 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:59.780168 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545c6f5-4a43-4a38-9e54-28559eb4a1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:59.780587 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545c6f5-4a43-4a38-9e54-28559eb4a1a4 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:59.781910 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 101/209] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:59.787707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 109/210] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:59.788977 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194ccc7-3664-49aa-9ef4-075e84f2f9aa None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:59.789468 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d729036-0730-4ff3-81c2-f9283e5f8847 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:59.789902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d729036-0730-4ff3-81c2-f9283e5f8847 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:59.790321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d729036-0730-4ff3-81c2-f9283e5f8847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:59.820646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-3d729036-0730-4ff3-81c2-f9283e5f8847 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:59.821851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3d729036-0730-4ff3-81c2-f9283e5f8847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:59.862754 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:59.902238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:59.925936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 102/211] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:59.927196 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfed40-accc-4aa0-a477-cdedb0a4b06d None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:59.927718 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfed40-accc-4aa0-a477-cdedb0a4b06d None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:59.927838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfed40-accc-4aa0-a477-cdedb0a4b06d None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:59.929158 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 103/212] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:59.935841 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7468c9-160a-4a15-b3fe-962d9c070d7c None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:59.936119 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7468c9-160a-4a15-b3fe-962d9c070d7c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:59.936393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7468c9-160a-4a15-b3fe-962d9c070d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:59.940911 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194ccc7-3664-49aa-9ef4-075e84f2f9aa None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:59.954899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-4f7468c9-160a-4a15-b3fe-962d9c070d7c None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:59.956198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4f7468c9-160a-4a15-b3fe-962d9c070d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:59.979336 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d14bd-10e4-4702-9b29-6a3fbea1c5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:59.979948 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d14bd-10e4-4702-9b29-6a3fbea1c5f4 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:59.980222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d14bd-10e4-4702-9b29-6a3fbea1c5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:59.980494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d14bd-10e4-4702-9b29-6a3fbea1c5f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:59.981222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d14bd-10e4-4702-9b29-6a3fbea1c5f4 None admin] RBAC: Adding query filter params (role.id=6c1ffb941f1646d2bf707005c7796790, scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69, user.id=3932ff5e04dd4aadbc9ba0548512f2be) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:59.981585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d14bd-10e4-4702-9b29-6a3fbea1c5f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:59.983120 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d14bd-10e4-4702-9b29-6a3fbea1c5f4 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:59.989909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 110/213] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:59 2022] GET /identity/v3/role_assignments?user.id=3932ff5e04dd4aadbc9ba0548512f2be&scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69&role.id=6c1ffb941f1646d2bf707005c7796790 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:00.044697 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 104/214] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:05:00.125029 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d14bd-10e4-4702-9b29-6a3fbea1c5f4 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:00.244041 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d14bd-10e4-4702-9b29-6a3fbea1c5f4 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:00.278899 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:00.288529 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4cb97-2198-406d-88b0-59099d7334e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:00.288529 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4cb97-2198-406d-88b0-59099d7334e1 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.288529 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4cb97-2198-406d-88b0-59099d7334e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.288529 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4cb97-2198-406d-88b0-59099d7334e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.301367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4cb97-2198-406d-88b0-59099d7334e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:00.307915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-7bd4cb97-2198-406d-88b0-59099d7334e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:00.307915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:00.307915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:00.307915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:00.307915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:00.307915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:00.307915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:00.307915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:00.307915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:00.307915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:00.307915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:00.307915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:00.307915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:00.307915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:00.307915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:00.307915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:00.313801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:00.313801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:00.313801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:00.313801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:00.313801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:00.313801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:00.313801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:00.313801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:00.313801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:00.313801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:00.313801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:00.313801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:00.313801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:00.313801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:00.313801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:00.313801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:00.314933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:00.314933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:00.314933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:00.314933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:00.314933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:00.314933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:00.314933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:00.314933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:00.314933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:00.314933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:00.314933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:00.314933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:00.314933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 111/215] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:00.316396 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7468c9-160a-4a15-b3fe-962d9c070d7c None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:00.317353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4cb97-2198-406d-88b0-59099d7334e1 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:00.431588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:00.442905 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4cb97-2198-406d-88b0-59099d7334e1 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:00.450511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7468c9-160a-4a15-b3fe-962d9c070d7c None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:00.480946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da642f94-bdce-4239-a6a3-41587ed1ea62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:00.481582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da642f94-bdce-4239-a6a3-41587ed1ea62 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.481858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da642f94-bdce-4239-a6a3-41587ed1ea62 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.482129 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da642f94-bdce-4239-a6a3-41587ed1ea62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.483251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da642f94-bdce-4239-a6a3-41587ed1ea62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:00.484690 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da642f94-bdce-4239-a6a3-41587ed1ea62 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:00.486430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 112/216] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:00.487116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da642f94-bdce-4239-a6a3-41587ed1ea62 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:00.492191 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b855999-6aa2-4a5a-b233-6cd879595e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:00.492808 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b855999-6aa2-4a5a-b233-6cd879595e0e None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.493085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b855999-6aa2-4a5a-b233-6cd879595e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.493352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b855999-6aa2-4a5a-b233-6cd879595e0e None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.494025 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b855999-6aa2-4a5a-b233-6cd879595e0e None admin] RBAC: Adding query filter params (name=member) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:00.494403 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b855999-6aa2-4a5a-b233-6cd879595e0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:00.495757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b855999-6aa2-4a5a-b233-6cd879595e0e None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:00.505315 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 105/217] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:00.506685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b855999-6aa2-4a5a-b233-6cd879595e0e None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:00.565584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:00.611285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da642f94-bdce-4239-a6a3-41587ed1ea62 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:00.616602 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b855999-6aa2-4a5a-b233-6cd879595e0e None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:00.644527 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e29d7-d885-41f6-a219-92bf6323b497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:00.645163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e29d7-d885-41f6-a219-92bf6323b497 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.645442 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e29d7-d885-41f6-a219-92bf6323b497 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.645715 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e29d7-d885-41f6-a219-92bf6323b497 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.651304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10150a05-52d6-4f27-b2e0-4b55e66e2ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:00.651915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10150a05-52d6-4f27-b2e0-4b55e66e2ccc None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.652198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10150a05-52d6-4f27-b2e0-4b55e66e2ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.652479 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10150a05-52d6-4f27-b2e0-4b55e66e2ccc None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.653132 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10150a05-52d6-4f27-b2e0-4b55e66e2ccc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:00.654576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10150a05-52d6-4f27-b2e0-4b55e66e2ccc None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:00.656222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e29d7-d885-41f6-a219-92bf6323b497 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:00.661491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-328e29d7-d885-41f6-a219-92bf6323b497 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:00.661491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:00.661491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:00.661491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:00.661491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:00.661491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:00.661491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:00.661491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:00.661491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:00.661491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:00.661491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:00.661491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:00.661491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:00.661491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:00.661491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:00.661491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:00.663136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:00.663136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:00.663136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:00.663136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:00.663136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:00.663136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:00.663136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:00.663136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:00.663136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:00.663136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:00.663136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:00.663136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:00.663136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:00.663136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:00.663136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:00.663136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:00.663968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:00.663968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:00.663968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:00.663968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:00.663968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:00.663968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:00.663968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:00.663968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:00.663968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:00.663968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:00.663968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:00.663968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:00.663968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 113/218] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:05:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:00.665571 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e29d7-d885-41f6-a219-92bf6323b497 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:00.717952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 106/219] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:05:00 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:00.719244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d705b0-669d-4172-81a1-80c46dbe6ae1 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.719537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d705b0-669d-4172-81a1-80c46dbe6ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.719813 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d705b0-669d-4172-81a1-80c46dbe6ae1 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.721060 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 107/220] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:00.721790 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d705b0-669d-4172-81a1-80c46dbe6ae1 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:00.732737 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:00.788480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e29d7-d885-41f6-a219-92bf6323b497 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:00.809660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:00.831759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5e944-401d-4df9-9263-eabdb9b356ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:00.832494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5e944-401d-4df9-9263-eabdb9b356ec None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.833314 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5e944-401d-4df9-9263-eabdb9b356ec None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.834127 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5e944-401d-4df9-9263-eabdb9b356ec None admin] PATH_INFO: `/v3/users/386d8bd3710341e785d489573294de18` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.850893 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5e944-401d-4df9-9263-eabdb9b356ec None admin] RBAC: Authorizing `identity:get_user(user_id=386d8bd3710341e785d489573294de18)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:00.852465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5e944-401d-4df9-9263-eabdb9b356ec None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:00.854219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 114/221] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:00 2022] GET /identity/v3/users/386d8bd3710341e785d489573294de18 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:00.855555 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99c977-595d-4610-ba73-2878d575ca03 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.855845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99c977-595d-4610-ba73-2878d575ca03 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.856123 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99c977-595d-4610-ba73-2878d575ca03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.870329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d705b0-669d-4172-81a1-80c46dbe6ae1 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:00.874776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-0b99c977-595d-4610-ba73-2878d575ca03 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:00.875999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-0b99c977-595d-4610-ba73-2878d575ca03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:00.914964 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62c7e2-319f-48a4-8f51-aaac4b51f5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:00.915591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62c7e2-319f-48a4-8f51-aaac4b51f5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.915879 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62c7e2-319f-48a4-8f51-aaac4b51f5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.916149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62c7e2-319f-48a4-8f51-aaac4b51f5e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.916939 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62c7e2-319f-48a4-8f51-aaac4b51f5e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:00.917243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62c7e2-319f-48a4-8f51-aaac4b51f5e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:00.918745 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62c7e2-319f-48a4-8f51-aaac4b51f5e6 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:00.928987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 108/222] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:00.929670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62c7e2-319f-48a4-8f51-aaac4b51f5e6 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:00.963737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 115/223] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:00.971698 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99c977-595d-4610-ba73-2878d575ca03 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.009068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:01.061608 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62c7e2-319f-48a4-8f51-aaac4b51f5e6 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:01.106957 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99c977-595d-4610-ba73-2878d575ca03 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:01.108601 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2875b0d-8145-467d-b7f1-bd6da5033099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.109309 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2875b0d-8145-467d-b7f1-bd6da5033099 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.109665 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2875b0d-8145-467d-b7f1-bd6da5033099 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.110735 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2875b0d-8145-467d-b7f1-bd6da5033099 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.135255 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:01.147731 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e45658-a812-403e-ac22-c2884487147b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.148649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e45658-a812-403e-ac22-c2884487147b None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.149082 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e45658-a812-403e-ac22-c2884487147b None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.149555 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e45658-a812-403e-ac22-c2884487147b None admin] PATH_INFO: `/v3/users/50d15c2581a04b59995fa2eb734bacc4` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.151967 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e45658-a812-403e-ac22-c2884487147b None admin] RBAC: Authorizing `identity:get_user(user_id=50d15c2581a04b59995fa2eb734bacc4)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:01.153613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e45658-a812-403e-ac22-c2884487147b None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:01.155419 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 116/224] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:00 2022] GET /identity/v3/users/50d15c2581a04b59995fa2eb734bacc4 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:01.160974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e45658-a812-403e-ac22-c2884487147b None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.198741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2875b0d-8145-467d-b7f1-bd6da5033099 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:01.201112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2875b0d-8145-467d-b7f1-bd6da5033099 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:01.283773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:01.289491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e45658-a812-403e-ac22-c2884487147b None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:01.329485 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01087a4-7eb8-4437-887e-d5cb492b77a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.330241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01087a4-7eb8-4437-887e-d5cb492b77a6 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.330625 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01087a4-7eb8-4437-887e-d5cb492b77a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.331052 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01087a4-7eb8-4437-887e-d5cb492b77a6 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.332412 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01087a4-7eb8-4437-887e-d5cb492b77a6 None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:01.334065 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01087a4-7eb8-4437-887e-d5cb492b77a6 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:01.336065 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 117/225] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:01 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:01.336217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 109/226] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:01.337099 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01087a4-7eb8-4437-887e-d5cb492b77a6 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.346906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2875b0d-8145-467d-b7f1-bd6da5033099 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.389740 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:01.453146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01087a4-7eb8-4437-887e-d5cb492b77a6 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:01.469122 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2875b0d-8145-467d-b7f1-bd6da5033099 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:01.493245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55606b9f-f4f6-4992-80c7-e27878fa0b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.493987 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55606b9f-f4f6-4992-80c7-e27878fa0b18 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.494342 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55606b9f-f4f6-4992-80c7-e27878fa0b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.494691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55606b9f-f4f6-4992-80c7-e27878fa0b18 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.506671 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55606b9f-f4f6-4992-80c7-e27878fa0b18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:01.511871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-55606b9f-f4f6-4992-80c7-e27878fa0b18 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:01.511871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:01.511871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:01.511871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:01.511871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:01.511871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:01.511871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:01.511871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:01.511871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:01.511871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:01.511871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:01.511871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:01.511871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:01.511871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:01.511871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:01.511871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:01.512768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:01.512768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:01.512768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:01.512768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:01.512768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:01.512768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:01.512768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:01.512768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:01.512768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:01.512768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:01.512768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:01.512768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:01.512768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:01.512768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:01.512768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:01.512768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:01.513568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:01.513568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:01.513568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:01.513568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:01.513568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:01.513568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:01.513568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:01.513568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:01.513568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:01.513568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:01.513568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:01.513568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:01.513568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 118/227] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:01.514251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55606b9f-f4f6-4992-80c7-e27878fa0b18 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.520953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09041ebd-60b9-4ff2-a9c2-63cc77704fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.523387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09041ebd-60b9-4ff2-a9c2-63cc77704fe7 None admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.523387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09041ebd-60b9-4ff2-a9c2-63cc77704fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.523387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09041ebd-60b9-4ff2-a9c2-63cc77704fe7 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d/users/50d15c2581a04b59995fa2eb734bacc4/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.525174 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09041ebd-60b9-4ff2-a9c2-63cc77704fe7 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca336d25ed0f4c00b1759ff3cac5302d, user_id=50d15c2581a04b59995fa2eb734bacc4, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:01.527183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09041ebd-60b9-4ff2-a9c2-63cc77704fe7 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:01.600173 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 110/228] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:05:01 2022] PUT /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d/users/50d15c2581a04b59995fa2eb734bacc4/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:05:01.603874 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09041ebd-60b9-4ff2-a9c2-63cc77704fe7 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.656446 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55606b9f-f4f6-4992-80c7-e27878fa0b18 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:01.659489 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:01.662850 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:01.731407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27e118-232b-483d-af1c-c16c353854bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.732062 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27e118-232b-483d-af1c-c16c353854bd None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.732344 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27e118-232b-483d-af1c-c16c353854bd None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.732622 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27e118-232b-483d-af1c-c16c353854bd None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.734073 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27e118-232b-483d-af1c-c16c353854bd None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:01.735565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27e118-232b-483d-af1c-c16c353854bd None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:01.737393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 119/229] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:01 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:01.745581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27e118-232b-483d-af1c-c16c353854bd None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.816679 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09041ebd-60b9-4ff2-a9c2-63cc77704fe7 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:01.856822 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d29ec3-b8da-4e58-91db-aff287f306eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.857373 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d29ec3-b8da-4e58-91db-aff287f306eb None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.857657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d29ec3-b8da-4e58-91db-aff287f306eb None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.858020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d29ec3-b8da-4e58-91db-aff287f306eb None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.858940 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d29ec3-b8da-4e58-91db-aff287f306eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:01.859170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d29ec3-b8da-4e58-91db-aff287f306eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:01.860564 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d29ec3-b8da-4e58-91db-aff287f306eb None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:01.870820 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 111/230] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:01.876612 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d29ec3-b8da-4e58-91db-aff287f306eb None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.902311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27e118-232b-483d-af1c-c16c353854bd None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:01.978409 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718f068-4469-49dc-a759-1a87c2599b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.979192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718f068-4469-49dc-a759-1a87c2599b26 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.979799 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718f068-4469-49dc-a759-1a87c2599b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.980088 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718f068-4469-49dc-a759-1a87c2599b26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.980829 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718f068-4469-49dc-a759-1a87c2599b26 None admin] RBAC: Adding query filter params (role.id=357fdcf3417c45319b9d424a5ab4e9ca, scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d, user.id=386d8bd3710341e785d489573294de18) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:01.981143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718f068-4469-49dc-a759-1a87c2599b26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:01.982657 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718f068-4469-49dc-a759-1a87c2599b26 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:01.990940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 120/231] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:01 2022] GET /identity/v3/role_assignments?user.id=386d8bd3710341e785d489573294de18&scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d&role.id=357fdcf3417c45319b9d424a5ab4e9ca => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:02.043265 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:02.054972 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d29ec3-b8da-4e58-91db-aff287f306eb None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:02.112937 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09feb80-35ba-4329-beba-d59b64c9bf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:02.113831 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09feb80-35ba-4329-beba-d59b64c9bf1a None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:02.114064 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09feb80-35ba-4329-beba-d59b64c9bf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:02.114324 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09feb80-35ba-4329-beba-d59b64c9bf1a None admin] PATH_INFO: `/v3/users/50d15c2581a04b59995fa2eb734bacc4` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:02.116193 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09feb80-35ba-4329-beba-d59b64c9bf1a None admin] RBAC: Authorizing `identity:get_user(user_id=50d15c2581a04b59995fa2eb734bacc4)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:02.117686 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09feb80-35ba-4329-beba-d59b64c9bf1a None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:02.120704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 112/232] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:01 2022] GET /identity/v3/users/50d15c2581a04b59995fa2eb734bacc4 => generated 283 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:02.126661 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718f068-4469-49dc-a759-1a87c2599b26 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:02.169366 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:02.247188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718f068-4469-49dc-a759-1a87c2599b26 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:02.296056 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a257ab3-7c9c-4af8-b916-5175dbadbaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:02.296709 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a257ab3-7c9c-4af8-b916-5175dbadbaa0 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:02.296993 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a257ab3-7c9c-4af8-b916-5175dbadbaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:02.297272 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a257ab3-7c9c-4af8-b916-5175dbadbaa0 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:02.301662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a257ab3-7c9c-4af8-b916-5175dbadbaa0 None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:02.303201 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a257ab3-7c9c-4af8-b916-5175dbadbaa0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:02.305389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 121/233] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:02 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:02.311776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09feb80-35ba-4329-beba-d59b64c9bf1a None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:02.466724 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09feb80-35ba-4329-beba-d59b64c9bf1a None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:02.510413 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec110544-2a30-46cb-a577-50ccca2ef378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:02.511113 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec110544-2a30-46cb-a577-50ccca2ef378 None admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:02.511470 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec110544-2a30-46cb-a577-50ccca2ef378 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:02.511751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec110544-2a30-46cb-a577-50ccca2ef378 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d/users/50d15c2581a04b59995fa2eb734bacc4/roles/e14f37f4a3a24f8da0f757aabb1f39be` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:02.514003 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec110544-2a30-46cb-a577-50ccca2ef378 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca336d25ed0f4c00b1759ff3cac5302d, user_id=50d15c2581a04b59995fa2eb734bacc4, role_id=e14f37f4a3a24f8da0f757aabb1f39be)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:02.515501 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec110544-2a30-46cb-a577-50ccca2ef378 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:02.628006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 113/234] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:05:02 2022] PUT /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d/users/50d15c2581a04b59995fa2eb734bacc4/roles/e14f37f4a3a24f8da0f757aabb1f39be => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:05:03.152180 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c36c0-d113-40a8-a443-8c6ad36b4fb7 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:03.152725 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c36c0-d113-40a8-a443-8c6ad36b4fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:03.152817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c36c0-d113-40a8-a443-8c6ad36b4fb7 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:03.154282 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 122/235] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:03.159525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81442ba7-8e32-4669-a3c5-2240356a523e None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:03.159951 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81442ba7-8e32-4669-a3c5-2240356a523e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:03.160126 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81442ba7-8e32-4669-a3c5-2240356a523e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:03.163871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c633e44-3ccd-4c3b-aa88-e0abe73761a7 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:03.164161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c633e44-3ccd-4c3b-aa88-e0abe73761a7 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:03.164437 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c633e44-3ccd-4c3b-aa88-e0abe73761a7 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:03.165863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 123/236] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:03.170939 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc662d-771b-46b6-8b87-8a84b632b53f None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:03.171226 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc662d-771b-46b6-8b87-8a84b632b53f None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:03.171502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc662d-771b-46b6-8b87-8a84b632b53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:03.188442 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-81442ba7-8e32-4669-a3c5-2240356a523e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:03.189759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-81442ba7-8e32-4669-a3c5-2240356a523e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:03.190220 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-dddc662d-771b-46b6-8b87-8a84b632b53f None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:03.191413 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-dddc662d-771b-46b6-8b87-8a84b632b53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:03.210938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:03.312796 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 114/237] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:03.335784 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 124/238] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:05:03.644938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81442ba7-8e32-4669-a3c5-2240356a523e None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:03.717810 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc662d-771b-46b6-8b87-8a84b632b53f None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:03.779933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:03.793474 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81442ba7-8e32-4669-a3c5-2240356a523e None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:03.841387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccf175-d411-4ea8-af1b-0510aee235fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:03.842042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccf175-d411-4ea8-af1b-0510aee235fc None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:03.842321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccf175-d411-4ea8-af1b-0510aee235fc None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:03.842599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccf175-d411-4ea8-af1b-0510aee235fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:03.854378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ccf175-d411-4ea8-af1b-0510aee235fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:03.859400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-18ccf175-d411-4ea8-af1b-0510aee235fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:05:03.859400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:03.859400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:03.859400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:03.859400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:03.859400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:03.859400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:03.859400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:03.859400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:03.859400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:03.859400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:03.859400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:03.859400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:03.859400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:03.859400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:03.859400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:03.860320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:03.860320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:03.860320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:03.860320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:03.860320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:03.860320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:03.860320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:03.860320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:03.860320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:03.860320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:03.860320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:03.860320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:03.860320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:03.860320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:03.860320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:03.860320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:03.861137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:03.861137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:03.861137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:03.861137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:03.861137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:03.861137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:03.861137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:03.861137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:03.861137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:03.861137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:03.861137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:05:03.861137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:03.861137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 115/239] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:03.869281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccf175-d411-4ea8-af1b-0510aee235fc None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:03.881880 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:03.918149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc662d-771b-46b6-8b87-8a84b632b53f None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:03.954689 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7a514-5f45-4ef2-8dda-0d95debfe9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:03.955369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7a514-5f45-4ef2-8dda-0d95debfe9e1 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:03.955650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7a514-5f45-4ef2-8dda-0d95debfe9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:03.955920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7a514-5f45-4ef2-8dda-0d95debfe9e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:03.957557 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7a514-5f45-4ef2-8dda-0d95debfe9e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:03.958646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7a514-5f45-4ef2-8dda-0d95debfe9e1 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:03.961286 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 125/240] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:03.970896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7a514-5f45-4ef2-8dda-0d95debfe9e1 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:04.038637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccf175-d411-4ea8-af1b-0510aee235fc None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:04.076883 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7399a1b-3cf7-4b02-b116-2e0396803b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:04.077787 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7399a1b-3cf7-4b02-b116-2e0396803b76 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:04.078216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7399a1b-3cf7-4b02-b116-2e0396803b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:04.078661 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7399a1b-3cf7-4b02-b116-2e0396803b76 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:04.079509 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7399a1b-3cf7-4b02-b116-2e0396803b76 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:04.080169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7399a1b-3cf7-4b02-b116-2e0396803b76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:04.081713 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7399a1b-3cf7-4b02-b116-2e0396803b76 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:04.092278 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 116/241] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:04.104158 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7399a1b-3cf7-4b02-b116-2e0396803b76 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:04.149716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7a514-5f45-4ef2-8dda-0d95debfe9e1 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:04.151596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:04.183353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:04.186534 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fac5b4-fab4-435d-8413-73df746d5c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:04.187291 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fac5b4-fab4-435d-8413-73df746d5c17 None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:04.187743 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fac5b4-fab4-435d-8413-73df746d5c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:04.188181 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fac5b4-fab4-435d-8413-73df746d5c17 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:04.188975 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fac5b4-fab4-435d-8413-73df746d5c17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:04.190518 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fac5b4-fab4-435d-8413-73df746d5c17 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:04.215505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7399a1b-3cf7-4b02-b116-2e0396803b76 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:04.251986 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384fe64-6ebd-453e-95bb-7ec76a434783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:04.252626 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384fe64-6ebd-453e-95bb-7ec76a434783 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:04.252911 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384fe64-6ebd-453e-95bb-7ec76a434783 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:04.253188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384fe64-6ebd-453e-95bb-7ec76a434783 None admin] PATH_INFO: `/v3/users/24b84cadee624625b976c8cb24ee02b1` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:04.268556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384fe64-6ebd-453e-95bb-7ec76a434783 None admin] RBAC: Authorizing `identity:get_user(user_id=24b84cadee624625b976c8cb24ee02b1)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:04.271711 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384fe64-6ebd-453e-95bb-7ec76a434783 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:04.272974 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 117/242] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:04 2022] GET /identity/v3/users/24b84cadee624625b976c8cb24ee02b1 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:04.274099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f2d47-c5a6-4b1f-a793-ab9357df5794 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:04.274390 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f2d47-c5a6-4b1f-a793-ab9357df5794 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:04.274669 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f2d47-c5a6-4b1f-a793-ab9357df5794 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:04.279511 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 126/243] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:05:03 2022] POST /identity/v3/users => generated 293 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:05:04.279681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 118/244] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:04.281526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fac5b4-fab4-435d-8413-73df746d5c17 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:04.284708 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d053d-316c-4814-8375-1d724b865d7f None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:04.285014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d053d-316c-4814-8375-1d724b865d7f None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:04.285226 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d053d-316c-4814-8375-1d724b865d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:04.314583 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-562d053d-316c-4814-8375-1d724b865d7f None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:04.316323 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-562d053d-316c-4814-8375-1d724b865d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:04.402039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 119/245] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:04.403850 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3d13f-80ab-4ee0-8876-77e99d5651b4 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:04.404362 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3d13f-80ab-4ee0-8876-77e99d5651b4 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:04.404848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3d13f-80ab-4ee0-8876-77e99d5651b4 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:04.406407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 120/246] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:04.412517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007785e-d793-4986-b6d5-04d09c400d64 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:04.413372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007785e-d793-4986-b6d5-04d09c400d64 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:04.413975 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007785e-d793-4986-b6d5-04d09c400d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:04.444002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-c007785e-d793-4986-b6d5-04d09c400d64 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:04.445510 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c007785e-d793-4986-b6d5-04d09c400d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:04.446084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fac5b4-fab4-435d-8413-73df746d5c17 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:04.489645 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56d5f5-5615-48ce-81f7-b1e39e436fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:04.490303 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56d5f5-5615-48ce-81f7-b1e39e436fbd None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:04.490577 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56d5f5-5615-48ce-81f7-b1e39e436fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:04.490847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56d5f5-5615-48ce-81f7-b1e39e436fbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:04.517553 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:04.523158 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:04.543825 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 121/247] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:04.576061 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56d5f5-5615-48ce-81f7-b1e39e436fbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:04.577860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56d5f5-5615-48ce-81f7-b1e39e436fbd None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:04.686027 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 127/248] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:04.691911 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007785e-d793-4986-b6d5-04d09c400d64 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:04.717807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56d5f5-5615-48ce-81f7-b1e39e436fbd None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:04.791666 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:04.845836 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56d5f5-5615-48ce-81f7-b1e39e436fbd None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:04.848551 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007785e-d793-4986-b6d5-04d09c400d64 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:04.894348 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1afea7-00cd-47d7-9276-168610dbe41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:04.895021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1afea7-00cd-47d7-9276-168610dbe41c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:04.895300 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1afea7-00cd-47d7-9276-168610dbe41c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:04.895571 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1afea7-00cd-47d7-9276-168610dbe41c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:04.895699 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a9c05-bf92-4a71-b934-75a83e24d0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:04.896341 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a9c05-bf92-4a71-b934-75a83e24d0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:04.896700 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a9c05-bf92-4a71-b934-75a83e24d0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:04.896952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a9c05-bf92-4a71-b934-75a83e24d0c1 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:04.898810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a9c05-bf92-4a71-b934-75a83e24d0c1 None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:04.900387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a9c05-bf92-4a71-b934-75a83e24d0c1 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:04.902208 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 122/249] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:04 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:04.903489 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a9c05-bf92-4a71-b934-75a83e24d0c1 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:04.907760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1afea7-00cd-47d7-9276-168610dbe41c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:04.913902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-ae1afea7-00cd-47d7-9276-168610dbe41c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:04.913902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:04.913902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:04.913902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:04.913902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:04.913902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:04.913902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:04.913902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:04.913902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:04.913902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:04.913902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:04.913902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:04.913902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:04.913902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:04.913902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:04.913902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:04.914858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:04.914858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:04.914858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:04.914858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:04.914858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:04.914858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:04.914858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:04.914858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:04.914858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:04.914858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:04.914858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:04.914858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:04.914858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:04.914858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:04.914858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:04.914858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:04.915783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:04.915783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:04.915783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:04.915783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:04.915783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:04.915783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:04.915783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:04.915783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:04.915783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:04.915783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:04.915783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:04.915783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:04.915783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 128/250] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:05:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:04.916441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1afea7-00cd-47d7-9276-168610dbe41c None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:04.984266 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:05.043290 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a9c05-bf92-4a71-b934-75a83e24d0c1 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:05.055809 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1afea7-00cd-47d7-9276-168610dbe41c None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:05.076299 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:05.088748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e341e3d-0a09-46c3-88ef-011dd3491487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:05.089664 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e341e3d-0a09-46c3-88ef-011dd3491487 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:05.092040 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e341e3d-0a09-46c3-88ef-011dd3491487 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:05.092040 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e341e3d-0a09-46c3-88ef-011dd3491487 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:05.105616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a799e-a34c-4e0a-a8e5-e26d74ec711b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:05.106231 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a799e-a34c-4e0a-a8e5-e26d74ec711b None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:05.106507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a799e-a34c-4e0a-a8e5-e26d74ec711b None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:05.106775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a799e-a34c-4e0a-a8e5-e26d74ec711b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:05.107530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a799e-a34c-4e0a-a8e5-e26d74ec711b None admin] RBAC: Adding query filter params (role.id=6c1ffb941f1646d2bf707005c7796790, scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d, user.id=24b84cadee624625b976c8cb24ee02b1) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:05.107838 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a799e-a34c-4e0a-a8e5-e26d74ec711b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:05.109607 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a799e-a34c-4e0a-a8e5-e26d74ec711b None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:05.110724 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e341e3d-0a09-46c3-88ef-011dd3491487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:05.116233 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 129/251] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:04 2022] GET /identity/v3/role_assignments?user.id=24b84cadee624625b976c8cb24ee02b1&scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d&role.id=6c1ffb941f1646d2bf707005c7796790 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:05.117032 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a799e-a34c-4e0a-a8e5-e26d74ec711b None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:05.119032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-5e341e3d-0a09-46c3-88ef-011dd3491487 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:05.119032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:05.119032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:05.119032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:05.119032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:05.119032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:05.119032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:05.119032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:05.119032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:05.119032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:05.119032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:05.119032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:05.119032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:05.119032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:05.119032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:05.119032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:05.119943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:05.119943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:05.119943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:05.119943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:05.119943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:05.119943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:05.119943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:05.119943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:05.119943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:05.119943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:05.119943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:05.119943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:05.119943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:05.119943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:05.119943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:05.119943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:05.120733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:05.120733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:05.120733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:05.120733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:05.120733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:05.120733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:05.120733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:05.120733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:05.120733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:05.120733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:05.120733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:05.120733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:05.127699 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 123/252] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:05.136108 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e341e3d-0a09-46c3-88ef-011dd3491487 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:05.329155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a799e-a34c-4e0a-a8e5-e26d74ec711b None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:05.344614 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e341e3d-0a09-46c3-88ef-011dd3491487 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:05.367935 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa70d5-b297-450d-97cd-5f69bd9d51cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:05.368560 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa70d5-b297-450d-97cd-5f69bd9d51cd None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:05.368845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa70d5-b297-450d-97cd-5f69bd9d51cd None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:05.369124 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa70d5-b297-450d-97cd-5f69bd9d51cd None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:05.369252 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:05.369817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa70d5-b297-450d-97cd-5f69bd9d51cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:05.370113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa70d5-b297-450d-97cd-5f69bd9d51cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:05.371532 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa70d5-b297-450d-97cd-5f69bd9d51cd None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:05.381293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 130/253] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:05.382737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29188cfa-ba0a-4c62-afc0-c91f05765ed5 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:05.383922 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29188cfa-ba0a-4c62-afc0-c91f05765ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:05.383922 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29188cfa-ba0a-4c62-afc0-c91f05765ed5 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:05.385869 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668f8c0-5d81-4add-b3f1-2e94fa32a0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:05.386741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668f8c0-5d81-4add-b3f1-2e94fa32a0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:05.387236 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668f8c0-5d81-4add-b3f1-2e94fa32a0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:05.387713 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668f8c0-5d81-4add-b3f1-2e94fa32a0f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:05.388619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2668f8c0-5d81-4add-b3f1-2e94fa32a0f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:05.388789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 131/254] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:05.389270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2668f8c0-5d81-4add-b3f1-2e94fa32a0f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:05.389716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29188cfa-ba0a-4c62-afc0-c91f05765ed5 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:05.391332 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2668f8c0-5d81-4add-b3f1-2e94fa32a0f5 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:05.400760 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 124/255] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:05.406892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b46de3-3be8-4a3b-b723-871fb4682675 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:05.407538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b46de3-3be8-4a3b-b723-871fb4682675 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:05.408135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b46de3-3be8-4a3b-b723-871fb4682675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:05.428450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-23b46de3-3be8-4a3b-b723-871fb4682675 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:05.429971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-23b46de3-3be8-4a3b-b723-871fb4682675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:05.448366 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:05.511138 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 125/256] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:05.511935 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b46de3-3be8-4a3b-b723-871fb4682675 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:05.540398 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29188cfa-ba0a-4c62-afc0-c91f05765ed5 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:05.580285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fdbaf-8783-417a-b123-f6b6fcf43501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:05.581790 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fdbaf-8783-417a-b123-f6b6fcf43501 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:05.582424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fdbaf-8783-417a-b123-f6b6fcf43501 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:05.582908 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fdbaf-8783-417a-b123-f6b6fcf43501 None admin] PATH_INFO: `/v3/users/50d15c2581a04b59995fa2eb734bacc4` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:05.585027 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fdbaf-8783-417a-b123-f6b6fcf43501 None admin] RBAC: Authorizing `identity:get_user(user_id=50d15c2581a04b59995fa2eb734bacc4)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:05.586672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fdbaf-8783-417a-b123-f6b6fcf43501 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:05.589055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 132/257] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:05 2022] GET /identity/v3/users/50d15c2581a04b59995fa2eb734bacc4 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:05.594664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fdbaf-8783-417a-b123-f6b6fcf43501 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:05.679530 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b46de3-3be8-4a3b-b723-871fb4682675 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:05.681945 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:05.714989 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12504938-7e36-4644-b687-527400b536f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:05.715777 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12504938-7e36-4644-b687-527400b536f2 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:05.716064 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12504938-7e36-4644-b687-527400b536f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:05.716342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12504938-7e36-4644-b687-527400b536f2 None admin] PATH_INFO: `/v3/users/386d8bd3710341e785d489573294de18` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:05.717562 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12504938-7e36-4644-b687-527400b536f2 None admin] RBAC: Authorizing `identity:get_user(user_id=386d8bd3710341e785d489573294de18)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:05.719199 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12504938-7e36-4644-b687-527400b536f2 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:05.721193 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 126/258] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:05 2022] GET /identity/v3/users/386d8bd3710341e785d489573294de18 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:05.731992 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12504938-7e36-4644-b687-527400b536f2 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:05.837052 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fdbaf-8783-417a-b123-f6b6fcf43501 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:05.860676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:05.888021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12504938-7e36-4644-b687-527400b536f2 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:05.941885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d65fa0-814b-498b-bc67-84e1056873b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:05.942745 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d65fa0-814b-498b-bc67-84e1056873b4 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:05.943124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d65fa0-814b-498b-bc67-84e1056873b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:05.943409 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d65fa0-814b-498b-bc67-84e1056873b4 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:05.943884 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1010469-b5b6-4804-bd10-998253c334b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:05.944618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1010469-b5b6-4804-bd10-998253c334b0 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:05.944722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d65fa0-814b-498b-bc67-84e1056873b4 None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:05.944894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1010469-b5b6-4804-bd10-998253c334b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:05.945174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1010469-b5b6-4804-bd10-998253c334b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:05.946224 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d65fa0-814b-498b-bc67-84e1056873b4 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:05.948099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 127/259] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:05 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:05.953534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d65fa0-814b-498b-bc67-84e1056873b4 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:06.034615 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:06.053230 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1010469-b5b6-4804-bd10-998253c334b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:06.055605 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1010469-b5b6-4804-bd10-998253c334b0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:06.173026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 133/260] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:06.174663 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1010469-b5b6-4804-bd10-998253c334b0 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:06.191461 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d65fa0-814b-498b-bc67-84e1056873b4 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:06.210048 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:06.228609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248be0c-0026-4c98-82bd-5a89f3773128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:06.229253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248be0c-0026-4c98-82bd-5a89f3773128 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:06.229535 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248be0c-0026-4c98-82bd-5a89f3773128 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:06.229811 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248be0c-0026-4c98-82bd-5a89f3773128 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:06.240925 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248be0c-0026-4c98-82bd-5a89f3773128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:06.247892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-2248be0c-0026-4c98-82bd-5a89f3773128 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:06.247892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:06.247892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:06.247892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:06.247892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:06.247892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:06.247892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:06.247892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:06.247892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:06.247892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:06.247892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:06.247892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:06.247892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:06.247892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:06.247892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:06.247892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:06.248995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:06.248995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:06.248995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:06.248995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:06.248995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:06.248995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:06.248995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:06.248995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:06.248995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:06.248995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:06.248995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:06.248995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:06.248995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:06.248995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:06.248995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:06.248995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:06.249786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:06.249786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:06.249786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:06.249786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:06.249786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:06.249786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:06.249786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:06.249786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:06.249786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:06.249786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:06.249786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:06.249786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:06.249786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 128/261] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:06.250462 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248be0c-0026-4c98-82bd-5a89f3773128 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:06.395235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:06.403426 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248be0c-0026-4c98-82bd-5a89f3773128 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:06.429735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1010469-b5b6-4804-bd10-998253c334b0 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:06.452659 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a3d6b-ebf1-481a-9604-4a2e4fdbd629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:06.453312 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a3d6b-ebf1-481a-9604-4a2e4fdbd629 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:06.453594 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a3d6b-ebf1-481a-9604-4a2e4fdbd629 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:06.453873 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a3d6b-ebf1-481a-9604-4a2e4fdbd629 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:06.455326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a3d6b-ebf1-481a-9604-4a2e4fdbd629 None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:06.456782 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a3d6b-ebf1-481a-9604-4a2e4fdbd629 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:06.458986 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 129/262] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:06 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:06.462893 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a3d6b-ebf1-481a-9604-4a2e4fdbd629 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:06.496448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4237d-ae78-4417-b982-c9576465aff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:06.497547 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4237d-ae78-4417-b982-c9576465aff0 None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:06.498049 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4237d-ae78-4417-b982-c9576465aff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:06.498527 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4237d-ae78-4417-b982-c9576465aff0 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d/users/386d8bd3710341e785d489573294de18/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:06.500731 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4237d-ae78-4417-b982-c9576465aff0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca336d25ed0f4c00b1759ff3cac5302d, user_id=386d8bd3710341e785d489573294de18, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:06.502402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4237d-ae78-4417-b982-c9576465aff0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:06.573176 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a3d6b-ebf1-481a-9604-4a2e4fdbd629 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:06.609254 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0864d18-db81-4285-b7cf-32702da78500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:06.609872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0864d18-db81-4285-b7cf-32702da78500 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:06.610158 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0864d18-db81-4285-b7cf-32702da78500 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:06.610435 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0864d18-db81-4285-b7cf-32702da78500 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:06.611144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0864d18-db81-4285-b7cf-32702da78500 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:06.611454 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0864d18-db81-4285-b7cf-32702da78500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:06.612846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0864d18-db81-4285-b7cf-32702da78500 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:06.623592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 130/263] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:06.624428 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0864d18-db81-4285-b7cf-32702da78500 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:06.657013 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:06.667504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 134/264] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:05:06 2022] PUT /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d/users/386d8bd3710341e785d489573294de18/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:05:06.668620 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4237d-ae78-4417-b982-c9576465aff0 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:06.786398 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0864d18-db81-4285-b7cf-32702da78500 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:06.842742 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fed13-7462-4e30-8ae4-6d5dda4f33d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:06.843412 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fed13-7462-4e30-8ae4-6d5dda4f33d5 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:06.843698 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fed13-7462-4e30-8ae4-6d5dda4f33d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:06.843976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fed13-7462-4e30-8ae4-6d5dda4f33d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:06.844751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75fed13-7462-4e30-8ae4-6d5dda4f33d5 None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d, user.id=50d15c2581a04b59995fa2eb734bacc4) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:06.846464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75fed13-7462-4e30-8ae4-6d5dda4f33d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:06.846890 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75fed13-7462-4e30-8ae4-6d5dda4f33d5 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:06.848418 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:06.856175 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 131/265] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:06 2022] GET /identity/v3/role_assignments?user.id=50d15c2581a04b59995fa2eb734bacc4&scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:06.915068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4237d-ae78-4417-b982-c9576465aff0 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:06.958197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed3469-f29b-4b5b-98aa-6268c719a61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:06.959827 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed3469-f29b-4b5b-98aa-6268c719a61d None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:06.960317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed3469-f29b-4b5b-98aa-6268c719a61d None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:06.960761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed3469-f29b-4b5b-98aa-6268c719a61d None admin] PATH_INFO: `/v3/users/50d15c2581a04b59995fa2eb734bacc4` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:06.962212 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed3469-f29b-4b5b-98aa-6268c719a61d None admin] RBAC: Authorizing `identity:get_user(user_id=50d15c2581a04b59995fa2eb734bacc4)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:06.963940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed3469-f29b-4b5b-98aa-6268c719a61d None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:06.965850 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 135/266] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:06 2022] GET /identity/v3/users/50d15c2581a04b59995fa2eb734bacc4 => generated 283 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:06.972157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fed13-7462-4e30-8ae4-6d5dda4f33d5 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:07.106258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:07.112996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fed13-7462-4e30-8ae4-6d5dda4f33d5 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:07.151174 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae7792-d1bc-4bf2-99dc-70376f8beb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:07.151859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae7792-d1bc-4bf2-99dc-70376f8beb96 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.152143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae7792-d1bc-4bf2-99dc-70376f8beb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.152466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae7792-d1bc-4bf2-99dc-70376f8beb96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.169818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472b974-0c33-4631-80e7-7687dda6d1e1 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.170124 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472b974-0c33-4631-80e7-7687dda6d1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.170373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472b974-0c33-4631-80e7-7687dda6d1e1 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.171997 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 136/267] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:07.178992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80984229-b200-46e9-b7d3-d5a043c2eaca None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.179570 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80984229-b200-46e9-b7d3-d5a043c2eaca None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.180055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80984229-b200-46e9-b7d3-d5a043c2eaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.204020 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-80984229-b200-46e9-b7d3-d5a043c2eaca None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:07.206133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-80984229-b200-46e9-b7d3-d5a043c2eaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:07.249031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae7792-d1bc-4bf2-99dc-70376f8beb96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:07.250500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae7792-d1bc-4bf2-99dc-70376f8beb96 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:07.304875 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 137/268] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:05:07.430604 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:07.438169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 132/269] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:07.444200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80984229-b200-46e9-b7d3-d5a043c2eaca None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:07.532655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:07.568482 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09ac90-47a8-4a91-b5b8-e8c23cc1aa03 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.568766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09ac90-47a8-4a91-b5b8-e8c23cc1aa03 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.569029 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09ac90-47a8-4a91-b5b8-e8c23cc1aa03 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.570529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 133/270] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:07.574166 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80984229-b200-46e9-b7d3-d5a043c2eaca None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:07.575575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea8fc1-ab4a-46fa-bd1e-cbd03b08c02c None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.575857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea8fc1-ab4a-46fa-bd1e-cbd03b08c02c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.576183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea8fc1-ab4a-46fa-bd1e-cbd03b08c02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.606807 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-71ea8fc1-ab4a-46fa-bd1e-cbd03b08c02c None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:07.608076 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-71ea8fc1-ab4a-46fa-bd1e-cbd03b08c02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:07.644482 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c5ac3-9ad0-41d7-a6ba-694dabe473c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:07.645140 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c5ac3-9ad0-41d7-a6ba-694dabe473c1 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.645419 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c5ac3-9ad0-41d7-a6ba-694dabe473c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.646184 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c5ac3-9ad0-41d7-a6ba-694dabe473c1 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.647525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c5ac3-9ad0-41d7-a6ba-694dabe473c1 None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:07.648962 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c5ac3-9ad0-41d7-a6ba-694dabe473c1 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:07.650773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 138/271] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:07 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:07.659744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c5ac3-9ad0-41d7-a6ba-694dabe473c1 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:07.700831 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 134/272] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:07.718838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea8fc1-ab4a-46fa-bd1e-cbd03b08c02c None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:07.842968 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea8fc1-ab4a-46fa-bd1e-cbd03b08c02c None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:07.868565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c5ac3-9ad0-41d7-a6ba-694dabe473c1 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:07.874513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:07.886576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e588e-aca5-487e-8a5c-5146650f9ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:07.887448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e588e-aca5-487e-8a5c-5146650f9ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.887726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e588e-aca5-487e-8a5c-5146650f9ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.888355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e588e-aca5-487e-8a5c-5146650f9ed6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.889761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e588e-aca5-487e-8a5c-5146650f9ed6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:07.891922 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e588e-aca5-487e-8a5c-5146650f9ed6 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:07.893883 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 135/273] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:07.895948 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3419a-55ca-4c66-bd87-664cb1bf3a86 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.896225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3419a-55ca-4c66-bd87-664cb1bf3a86 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.896616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3419a-55ca-4c66-bd87-664cb1bf3a86 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.898849 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 136/274] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:07.900095 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3419a-55ca-4c66-bd87-664cb1bf3a86 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:07.916371 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5cd4a-40e5-4b0f-844c-5e506202025c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:07.917140 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5cd4a-40e5-4b0f-844c-5e506202025c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.917537 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5cd4a-40e5-4b0f-844c-5e506202025c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.917821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5cd4a-40e5-4b0f-844c-5e506202025c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.918549 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad5cd4a-40e5-4b0f-844c-5e506202025c None admin] RBAC: Adding query filter params (role.id=e14f37f4a3a24f8da0f757aabb1f39be, scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d, user.id=50d15c2581a04b59995fa2eb734bacc4) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:07.919124 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad5cd4a-40e5-4b0f-844c-5e506202025c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:07.921028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad5cd4a-40e5-4b0f-844c-5e506202025c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:07.925029 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:07.930406 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 139/275] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:07 2022] GET /identity/v3/role_assignments?user.id=50d15c2581a04b59995fa2eb734bacc4&scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d&role.id=e14f37f4a3a24f8da0f757aabb1f39be => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:07.932651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f136d74-d633-4655-8967-a9c37066740e None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.933236 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f136d74-d633-4655-8967-a9c37066740e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.933650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f136d74-d633-4655-8967-a9c37066740e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.959519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-4f136d74-d633-4655-8967-a9c37066740e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:07.961569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4f136d74-d633-4655-8967-a9c37066740e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:08.057065 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3419a-55ca-4c66-bd87-664cb1bf3a86 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:08.074116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 140/276] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:08.075107 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f136d74-d633-4655-8967-a9c37066740e None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:08.113803 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad94a9a-a6dd-4f9e-9660-93ef3bfd487e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:08.115522 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad94a9a-a6dd-4f9e-9660-93ef3bfd487e None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:08.116030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad94a9a-a6dd-4f9e-9660-93ef3bfd487e None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:08.116482 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad94a9a-a6dd-4f9e-9660-93ef3bfd487e None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:08.117306 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad94a9a-a6dd-4f9e-9660-93ef3bfd487e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:08.119118 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad94a9a-a6dd-4f9e-9660-93ef3bfd487e None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:08.216884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 137/277] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:05:07 2022] POST /identity/v3/users => generated 293 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:08.228855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f136d74-d633-4655-8967-a9c37066740e None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:08.267693 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:08.278382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53005c1a-501a-4e7a-8f40-84dd2256fd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:08.279092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53005c1a-501a-4e7a-8f40-84dd2256fd82 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:08.279375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53005c1a-501a-4e7a-8f40-84dd2256fd82 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:08.279706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53005c1a-501a-4e7a-8f40-84dd2256fd82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:08.298632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53005c1a-501a-4e7a-8f40-84dd2256fd82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:08.301589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-53005c1a-501a-4e7a-8f40-84dd2256fd82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:08.301589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:08.301589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:08.301589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:08.301589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:08.301589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:08.301589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:08.301589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:08.301589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:08.301589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:08.301589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:08.301589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:08.301589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:08.301589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:08.301589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:08.301589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:08.302840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:08.302840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:08.302840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:08.302840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:08.302840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:08.302840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:08.302840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:08.302840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:08.302840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:08.302840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:08.302840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:08.302840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:08.302840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:08.302840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:08.302840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:08.302840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:08.303670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:08.303670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:08.303670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:08.303670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:08.303670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:08.303670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:08.303670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:08.303670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:08.303670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:08.303670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:08.303670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:08.303670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:08.306988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 141/278] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:08.309436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad94a9a-a6dd-4f9e-9660-93ef3bfd487e None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:08.367762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:08.394703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53005c1a-501a-4e7a-8f40-84dd2256fd82 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:08.432713 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad94a9a-a6dd-4f9e-9660-93ef3bfd487e None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:08.474212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425900b-40ca-4b2d-937a-3b9211daa241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:08.474946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425900b-40ca-4b2d-937a-3b9211daa241 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:08.475240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425900b-40ca-4b2d-937a-3b9211daa241 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:08.475520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425900b-40ca-4b2d-937a-3b9211daa241 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:08.476230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425900b-40ca-4b2d-937a-3b9211daa241 None admin] RBAC: Adding query filter params (name=member) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:08.476541 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425900b-40ca-4b2d-937a-3b9211daa241 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:08.477984 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425900b-40ca-4b2d-937a-3b9211daa241 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:08.494165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 138/279] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:08.500010 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425900b-40ca-4b2d-937a-3b9211daa241 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:08.529714 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53005c1a-501a-4e7a-8f40-84dd2256fd82 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:08.574425 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839dec4-d11a-492f-8b13-19a526621eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:08.576042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839dec4-d11a-492f-8b13-19a526621eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:08.576042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839dec4-d11a-492f-8b13-19a526621eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:08.576042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839dec4-d11a-492f-8b13-19a526621eb8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:08.594994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839dec4-d11a-492f-8b13-19a526621eb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:08.602603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-1839dec4-d11a-492f-8b13-19a526621eb8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:05:08.602603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:08.602603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:08.602603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:08.602603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:08.602603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:08.602603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:08.602603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:08.602603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:08.602603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:08.602603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:08.602603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:08.602603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:08.602603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:08.602603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:08.602603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:08.603616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:08.603616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:08.603616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:08.603616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:08.603616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:08.603616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:08.603616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:08.603616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:08.603616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:08.603616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:08.603616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:08.603616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:08.603616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:08.603616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:08.603616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:08.603616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:08.604432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:08.604432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:08.604432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:08.604432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:08.604432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:08.604432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:08.604432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:08.604432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:08.604432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:08.604432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:08.604432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:05:08.604432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:08.604432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 142/280] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:08.610845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839dec4-d11a-492f-8b13-19a526621eb8 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:08.638025 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425900b-40ca-4b2d-937a-3b9211daa241 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:08.650206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:08.672954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:08.684822 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c4955-e7cd-4af2-becf-01b7de5d4174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:08.685709 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c4955-e7cd-4af2-becf-01b7de5d4174 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:08.686054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c4955-e7cd-4af2-becf-01b7de5d4174 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:08.686396 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c4955-e7cd-4af2-becf-01b7de5d4174 None admin] PATH_INFO: `/v3/users/6e46aede8b6347c088c36c65bdc84fff` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:08.705650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c4955-e7cd-4af2-becf-01b7de5d4174 None admin] RBAC: Authorizing `identity:get_user(user_id=6e46aede8b6347c088c36c65bdc84fff)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:08.707872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c4955-e7cd-4af2-becf-01b7de5d4174 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:08.710730 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 139/281] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:08 2022] GET /identity/v3/users/6e46aede8b6347c088c36c65bdc84fff => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:08.716421 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c4955-e7cd-4af2-becf-01b7de5d4174 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:08.743903 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839dec4-d11a-492f-8b13-19a526621eb8 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:08.790522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683af0e-150c-4465-91b0-12894bbd136f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:08.791302 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683af0e-150c-4465-91b0-12894bbd136f None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:08.791642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683af0e-150c-4465-91b0-12894bbd136f None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:08.791981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683af0e-150c-4465-91b0-12894bbd136f None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:08.793050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683af0e-150c-4465-91b0-12894bbd136f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:08.793417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683af0e-150c-4465-91b0-12894bbd136f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:08.795556 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683af0e-150c-4465-91b0-12894bbd136f None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:08.807202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 143/282] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:08.815511 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683af0e-150c-4465-91b0-12894bbd136f None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:08.843891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c4955-e7cd-4af2-becf-01b7de5d4174 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:08.892624 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8f8b5-1f28-4822-a2cd-c0520e697a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:08.893395 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8f8b5-1f28-4822-a2cd-c0520e697a73 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:08.893928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8f8b5-1f28-4822-a2cd-c0520e697a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:08.894299 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8f8b5-1f28-4822-a2cd-c0520e697a73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:08.895493 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8f8b5-1f28-4822-a2cd-c0520e697a73 None admin] RBAC: Adding query filter params (role.id=357fdcf3417c45319b9d424a5ab4e9ca, user.id=6e46aede8b6347c088c36c65bdc84fff, scope.system=all) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:08.896607 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8f8b5-1f28-4822-a2cd-c0520e697a73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:08.898464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8f8b5-1f28-4822-a2cd-c0520e697a73 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:08.905644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 140/283] 10.222.0.59 () {56 vars in 1308 bytes} [Wed Aug 31 19:05:08 2022] GET /identity/v3/role_assignments?user.id=6e46aede8b6347c088c36c65bdc84fff&scope.system=all&role.id=357fdcf3417c45319b9d424a5ab4e9ca => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:08.949129 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683af0e-150c-4465-91b0-12894bbd136f None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:08.963922 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:08.996439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74b9c9-fee3-4e99-93ca-0f3f39e687c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:08.997018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74b9c9-fee3-4e99-93ca-0f3f39e687c1 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:08.997296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74b9c9-fee3-4e99-93ca-0f3f39e687c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:08.997567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74b9c9-fee3-4e99-93ca-0f3f39e687c1 None admin] PATH_INFO: `/v3/users/24b84cadee624625b976c8cb24ee02b1` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:08.998711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74b9c9-fee3-4e99-93ca-0f3f39e687c1 None admin] RBAC: Authorizing `identity:get_user(user_id=24b84cadee624625b976c8cb24ee02b1)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:09.000190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74b9c9-fee3-4e99-93ca-0f3f39e687c1 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:09.002136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 144/284] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:08 2022] GET /identity/v3/users/24b84cadee624625b976c8cb24ee02b1 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:09.007984 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8f8b5-1f28-4822-a2cd-c0520e697a73 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:09.061125 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:09.163909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8f8b5-1f28-4822-a2cd-c0520e697a73 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:09.187079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2aa15-4f8c-4a09-811a-e6ccb457653a None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:09.187709 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2aa15-4f8c-4a09-811a-e6ccb457653a None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:09.188250 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2aa15-4f8c-4a09-811a-e6ccb457653a None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:09.190414 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 145/285] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:09.196638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccf9b5-6fb6-439b-85a4-df665535e14b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:09.197267 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccf9b5-6fb6-439b-85a4-df665535e14b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:09.197771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccf9b5-6fb6-439b-85a4-df665535e14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:09.207825 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b3832-52d0-453b-88f2-6e1450e16c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:09.208519 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b3832-52d0-453b-88f2-6e1450e16c09 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:09.208805 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b3832-52d0-453b-88f2-6e1450e16c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:09.209176 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b3832-52d0-453b-88f2-6e1450e16c09 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:09.210357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b3832-52d0-453b-88f2-6e1450e16c09 None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:09.211839 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b3832-52d0-453b-88f2-6e1450e16c09 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:09.218241 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 141/286] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:09 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:09.220655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b3832-52d0-453b-88f2-6e1450e16c09 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:09.230311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-deccf9b5-6fb6-439b-85a4-df665535e14b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:09.236102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-deccf9b5-6fb6-439b-85a4-df665535e14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:09.368226 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 146/287] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:05:09.393621 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:09.401741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b3832-52d0-453b-88f2-6e1450e16c09 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:09.445187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9cde9-04d2-4110-aad4-ae4917f8c2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:09.445880 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9cde9-04d2-4110-aad4-ae4917f8c2ce None admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:09.446172 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9cde9-04d2-4110-aad4-ae4917f8c2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:09.446376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9cde9-04d2-4110-aad4-ae4917f8c2ce None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d/users/24b84cadee624625b976c8cb24ee02b1/roles/6c1ffb941f1646d2bf707005c7796790` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:09.448377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9cde9-04d2-4110-aad4-ae4917f8c2ce None admin] RBAC: Authorizing `identity:create_grant(project_id=ca336d25ed0f4c00b1759ff3cac5302d, user_id=24b84cadee624625b976c8cb24ee02b1, role_id=6c1ffb941f1646d2bf707005c7796790)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:09.449817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9cde9-04d2-4110-aad4-ae4917f8c2ce None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:09.524320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 142/288] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:05:09 2022] PUT /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d/users/24b84cadee624625b976c8cb24ee02b1/roles/6c1ffb941f1646d2bf707005c7796790 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:05:09.687877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccf9b5-6fb6-439b-85a4-df665535e14b None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:09.805684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccf9b5-6fb6-439b-85a4-df665535e14b None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:09.818751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:09.844290 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e3bdf-efa4-477d-a086-140db2a49014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:09.845286 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e3bdf-efa4-477d-a086-140db2a49014 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:09.845815 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e3bdf-efa4-477d-a086-140db2a49014 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:09.846285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e3bdf-efa4-477d-a086-140db2a49014 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:09.861164 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e3bdf-efa4-477d-a086-140db2a49014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:09.866368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-ac5e3bdf-efa4-477d-a086-140db2a49014 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:09.866368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:09.866368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:09.866368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:09.866368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:09.866368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:09.866368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:09.866368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:09.866368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:09.866368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:09.866368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:09.866368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:09.866368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:09.866368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:09.866368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:09.866368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:09.867296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:09.867296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:09.867296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:09.867296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:09.867296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:09.867296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:09.867296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:09.867296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:09.867296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:09.867296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:09.867296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:09.867296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:09.867296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:09.867296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:09.867296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:09.867296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:09.868676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:09.868676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:09.868676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:09.868676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:09.868676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:09.868676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:09.868676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:09.868676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:09.868676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:09.868676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:09.868676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:09.868676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:09.868676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 147/289] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:09.879041 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9cde9-04d2-4110-aad4-ae4917f8c2ce None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:10.025388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9cde9-04d2-4110-aad4-ae4917f8c2ce None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:10.074758 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316f74f-2efa-4adc-8d61-db22c16a9245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:10.075388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316f74f-2efa-4adc-8d61-db22c16a9245 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:10.075669 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316f74f-2efa-4adc-8d61-db22c16a9245 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:10.075938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316f74f-2efa-4adc-8d61-db22c16a9245 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:10.076604 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c316f74f-2efa-4adc-8d61-db22c16a9245 None admin] RBAC: Adding query filter params (name=member) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:10.076901 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c316f74f-2efa-4adc-8d61-db22c16a9245 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:10.078273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c316f74f-2efa-4adc-8d61-db22c16a9245 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:10.090896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 143/290] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:10.096503 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e3bdf-efa4-477d-a086-140db2a49014 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:10.264527 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e3bdf-efa4-477d-a086-140db2a49014 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:10.327769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d48358-eecd-45f3-b6b7-d8f5e7b9bc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:10.328422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d48358-eecd-45f3-b6b7-d8f5e7b9bc20 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:10.328707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d48358-eecd-45f3-b6b7-d8f5e7b9bc20 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:10.328980 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d48358-eecd-45f3-b6b7-d8f5e7b9bc20 None admin] PATH_INFO: `/v3/users/386d8bd3710341e785d489573294de18` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:10.335611 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d48358-eecd-45f3-b6b7-d8f5e7b9bc20 None admin] RBAC: Authorizing `identity:get_user(user_id=386d8bd3710341e785d489573294de18)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:10.337074 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d48358-eecd-45f3-b6b7-d8f5e7b9bc20 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:10.339057 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 148/291] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:10 2022] GET /identity/v3/users/386d8bd3710341e785d489573294de18 => generated 297 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:10.344313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316f74f-2efa-4adc-8d61-db22c16a9245 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:10.389138 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:10.484825 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316f74f-2efa-4adc-8d61-db22c16a9245 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:10.532331 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fb416-b5fc-44b1-bff3-713030326d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:10.533169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fb416-b5fc-44b1-bff3-713030326d7b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:10.533454 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fb416-b5fc-44b1-bff3-713030326d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:10.533734 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fb416-b5fc-44b1-bff3-713030326d7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:10.615044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711fb416-b5fc-44b1-bff3-713030326d7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:10.620754 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711fb416-b5fc-44b1-bff3-713030326d7b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:10.662631 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:10.739115 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 144/292] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:10.745162 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d48358-eecd-45f3-b6b7-d8f5e7b9bc20 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:10.813710 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:10.829986 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9848a59-ce0b-4a34-8c36-f69c5c00c97b None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:10.830325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9848a59-ce0b-4a34-8c36-f69c5c00c97b None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:10.830533 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9848a59-ce0b-4a34-8c36-f69c5c00c97b None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:10.832117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 145/293] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:10.837203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccfc2e-7422-4248-a083-5289452b04de None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:10.837554 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccfc2e-7422-4248-a083-5289452b04de None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:10.837827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccfc2e-7422-4248-a083-5289452b04de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:10.856799 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-e1ccfc2e-7422-4248-a083-5289452b04de None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:10.857993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e1ccfc2e-7422-4248-a083-5289452b04de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:10.933133 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 146/294] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:05:10.939884 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d48358-eecd-45f3-b6b7-d8f5e7b9bc20 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:10.995052 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c268bb-b08c-442e-9301-a3616ca3417c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:10.996075 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c268bb-b08c-442e-9301-a3616ca3417c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:10.996561 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c268bb-b08c-442e-9301-a3616ca3417c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:10.997040 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c268bb-b08c-442e-9301-a3616ca3417c None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.003551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c268bb-b08c-442e-9301-a3616ca3417c None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:11.005803 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c268bb-b08c-442e-9301-a3616ca3417c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:11.008089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 149/295] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:10 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:11.013601 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccfc2e-7422-4248-a083-5289452b04de None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:11.090519 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:11.155262 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccfc2e-7422-4248-a083-5289452b04de None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:11.204978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c268bb-b08c-442e-9301-a3616ca3417c None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:11.209301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371ae63-2e37-414d-9058-4eb5a5e72bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:11.209929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371ae63-2e37-414d-9058-4eb5a5e72bab None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.210213 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371ae63-2e37-414d-9058-4eb5a5e72bab None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.210488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371ae63-2e37-414d-9058-4eb5a5e72bab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.211252 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371ae63-2e37-414d-9058-4eb5a5e72bab None admin] RBAC: Adding query filter params (role.id=357fdcf3417c45319b9d424a5ab4e9ca, scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d, user.id=386d8bd3710341e785d489573294de18) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:11.211567 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371ae63-2e37-414d-9058-4eb5a5e72bab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:11.213024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371ae63-2e37-414d-9058-4eb5a5e72bab None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:11.224150 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 147/296] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:11 2022] GET /identity/v3/role_assignments?user.id=386d8bd3710341e785d489573294de18&scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d&role.id=357fdcf3417c45319b9d424a5ab4e9ca => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:11.244918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca28eba-cac8-46e1-b3b5-ed09b90cb744 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.245264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca28eba-cac8-46e1-b3b5-ed09b90cb744 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.245446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca28eba-cac8-46e1-b3b5-ed09b90cb744 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.247447 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 148/297] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:11.272916 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facc793-a138-45d7-a234-b9cf5663e38e None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.273216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facc793-a138-45d7-a234-b9cf5663e38e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.273654 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facc793-a138-45d7-a234-b9cf5663e38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.309382 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-0facc793-a138-45d7-a234-b9cf5663e38e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:11.310609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-0facc793-a138-45d7-a234-b9cf5663e38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:11.360718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c268bb-b08c-442e-9301-a3616ca3417c None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:11.376529 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:11.416694 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fae41-c2d6-461a-aab5-f2da5e8a7f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:11.417727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fae41-c2d6-461a-aab5-f2da5e8a7f33 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.418202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fae41-c2d6-461a-aab5-f2da5e8a7f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.418668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fae41-c2d6-461a-aab5-f2da5e8a7f33 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.434039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fae41-c2d6-461a-aab5-f2da5e8a7f33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:11.440528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-eb3fae41-c2d6-461a-aab5-f2da5e8a7f33 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:05:11.440528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:11.440528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:11.440528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:11.440528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:11.440528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:11.440528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:11.440528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:11.440528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:11.440528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:11.440528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:11.440528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:11.440528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:11.440528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:11.440528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:11.440528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:11.449243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:11.449243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:11.449243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:11.449243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:11.449243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:11.449243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:11.449243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:11.449243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:11.449243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:11.449243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:11.449243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:11.449243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:11.449243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:11.449243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:11.449243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:11.449243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:11.450095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:11.450095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:11.450095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:11.450095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:11.450095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:11.450095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:11.450095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:11.450095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:11.450095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:11.450095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:11.450095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:05:11.450095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:11.450095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 150/298] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:11.450768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 149/299] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:05:11.450848 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fae41-c2d6-461a-aab5-f2da5e8a7f33 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:11.532270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6631a18-5cfa-47fe-bac4-3063313f531c None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.532780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6631a18-5cfa-47fe-bac4-3063313f531c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.532780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6631a18-5cfa-47fe-bac4-3063313f531c None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.534275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 150/300] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:11.539948 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134c9ba-8042-4bda-ac57-5cb79b1836e9 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.540227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134c9ba-8042-4bda-ac57-5cb79b1836e9 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.540504 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134c9ba-8042-4bda-ac57-5cb79b1836e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.562676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-b134c9ba-8042-4bda-ac57-5cb79b1836e9 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:11.562676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-b134c9ba-8042-4bda-ac57-5cb79b1836e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:11.569800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fae41-c2d6-461a-aab5-f2da5e8a7f33 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:11.608310 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea49ca-eeec-4b4e-8f11-ba9bb2810d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:11.609306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea49ca-eeec-4b4e-8f11-ba9bb2810d05 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.609781 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea49ca-eeec-4b4e-8f11-ba9bb2810d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.610396 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:11.611515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea49ca-eeec-4b4e-8f11-ba9bb2810d05 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.612479 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea49ca-eeec-4b4e-8f11-ba9bb2810d05 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:11.612994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea49ca-eeec-4b4e-8f11-ba9bb2810d05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:11.615498 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea49ca-eeec-4b4e-8f11-ba9bb2810d05 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:11.624984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 151/301] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:11.631580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea49ca-eeec-4b4e-8f11-ba9bb2810d05 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:11.642098 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 151/302] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:05:11.665781 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:11.751421 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea49ca-eeec-4b4e-8f11-ba9bb2810d05 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:11.790958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ee31e-11eb-487e-8a7b-8d1d680f8ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:11.791918 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ee31e-11eb-487e-8a7b-8d1d680f8ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.792394 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ee31e-11eb-487e-8a7b-8d1d680f8ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.792861 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ee31e-11eb-487e-8a7b-8d1d680f8ac1 None admin] PATH_INFO: `/v3/users/4c976853b632408a98e320c398c99f13` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.807110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ee31e-11eb-487e-8a7b-8d1d680f8ac1 None admin] RBAC: Authorizing `identity:get_user(user_id=4c976853b632408a98e320c398c99f13)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:11.808873 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ee31e-11eb-487e-8a7b-8d1d680f8ac1 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:11.810690 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 152/303] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:11 2022] GET /identity/v3/users/4c976853b632408a98e320c398c99f13 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:11.817226 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134c9ba-8042-4bda-ac57-5cb79b1836e9 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:11.818492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ee31e-11eb-487e-8a7b-8d1d680f8ac1 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:11.919001 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:11.936664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ee31e-11eb-487e-8a7b-8d1d680f8ac1 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:11.950208 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134c9ba-8042-4bda-ac57-5cb79b1836e9 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:11.973222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400e9d4-0c4b-4b79-bdde-c4e1b3135563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:11.974473 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400e9d4-0c4b-4b79-bdde-c4e1b3135563 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.974750 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400e9d4-0c4b-4b79-bdde-c4e1b3135563 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.975069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400e9d4-0c4b-4b79-bdde-c4e1b3135563 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.976165 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400e9d4-0c4b-4b79-bdde-c4e1b3135563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:11.977599 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400e9d4-0c4b-4b79-bdde-c4e1b3135563 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:11.979185 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 153/304] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:11.980383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c39870-5544-4d3e-ad79-9114eb50d835 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.980664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c39870-5544-4d3e-ad79-9114eb50d835 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.980934 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c39870-5544-4d3e-ad79-9114eb50d835 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.982227 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 154/305] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:11.982782 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c39870-5544-4d3e-ad79-9114eb50d835 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:11.988020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0fbfa7-7ec4-4154-8abf-05f0bf919bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:11.989133 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fbfa7-7ec4-4154-8abf-05f0bf919bae None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.989589 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fbfa7-7ec4-4154-8abf-05f0bf919bae None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.989945 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fbfa7-7ec4-4154-8abf-05f0bf919bae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.990682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0fbfa7-7ec4-4154-8abf-05f0bf919bae None admin] RBAC: Adding query filter params (role.id=6c1ffb941f1646d2bf707005c7796790, user.id=4c976853b632408a98e320c398c99f13, scope.system=all) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:11.991027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0fbfa7-7ec4-4154-8abf-05f0bf919bae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:11.992906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0fbfa7-7ec4-4154-8abf-05f0bf919bae None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:11.998906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 152/306] 10.222.0.59 () {56 vars in 1308 bytes} [Wed Aug 31 19:05:11 2022] GET /identity/v3/role_assignments?user.id=4c976853b632408a98e320c398c99f13&scope.system=all&role.id=6c1ffb941f1646d2bf707005c7796790 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:11.999498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0fbfa7-7ec4-4154-8abf-05f0bf919bae None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:12.049844 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:12.111763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c39870-5544-4d3e-ad79-9114eb50d835 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:12.120923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0fbfa7-7ec4-4154-8abf-05f0bf919bae None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:12.157624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73f20c-6fcc-4b82-ae1f-c39d01bd079a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:12.159288 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73f20c-6fcc-4b82-ae1f-c39d01bd079a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:12.160442 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73f20c-6fcc-4b82-ae1f-c39d01bd079a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:12.161968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73f20c-6fcc-4b82-ae1f-c39d01bd079a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:12.166518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3e595-2c03-4b91-9b71-5914cc58de5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:12.167169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3e595-2c03-4b91-9b71-5914cc58de5a None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:12.167454 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3e595-2c03-4b91-9b71-5914cc58de5a None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:12.167731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3e595-2c03-4b91-9b71-5914cc58de5a None admin] PATH_INFO: `/v3/groups` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:12.168371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3e595-2c03-4b91-9b71-5914cc58de5a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:12.169845 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3e595-2c03-4b91-9b71-5914cc58de5a None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:12.175039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a73f20c-6fcc-4b82-ae1f-c39d01bd079a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:12.181431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-8a73f20c-6fcc-4b82-ae1f-c39d01bd079a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:12.181431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:12.181431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:12.181431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:12.181431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:12.181431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:12.181431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:12.181431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:12.181431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:12.181431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:12.181431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:12.181431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:12.181431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:12.181431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:12.181431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:12.181431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:12.182589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:12.182589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:12.182589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:12.182589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:12.182589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:12.182589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:12.182589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:12.182589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:12.182589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:12.182589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:12.182589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:12.182589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:12.182589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:12.182589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:12.182589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:12.182589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:12.183474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:12.183474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:12.183474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:12.183474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:12.183474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:12.183474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:12.183474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:12.183474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:12.183474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:12.183474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:12.183474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:12.183474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:12.183474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 155/307] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:12.184156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb626c1-d5b2-490b-98f3-a83406cd46cc None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:12.184384 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb626c1-d5b2-490b-98f3-a83406cd46cc None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:12.184656 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb626c1-d5b2-490b-98f3-a83406cd46cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:12.203129 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-cfb626c1-d5b2-490b-98f3-a83406cd46cc None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:12.204347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-cfb626c1-d5b2-490b-98f3-a83406cd46cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:12.216137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:12.245079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 153/308] 10.222.0.59 () {60 vars in 1134 bytes} [Wed Aug 31 19:05:11 2022] POST /identity/v3/groups => generated 229 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:12.246176 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3e595-2c03-4b91-9b71-5914cc58de5a None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:12.288656 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 156/309] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:12.391311 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:12.395510 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3e595-2c03-4b91-9b71-5914cc58de5a None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:12.434675 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa40bf-47fd-4f3c-9861-097d92dc2756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:12.435326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa40bf-47fd-4f3c-9861-097d92dc2756 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:12.435610 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa40bf-47fd-4f3c-9861-097d92dc2756 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:12.436060 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa40bf-47fd-4f3c-9861-097d92dc2756 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:12.436598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa40bf-47fd-4f3c-9861-097d92dc2756 None admin] RBAC: Adding query filter params (name=member) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:12.436929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa40bf-47fd-4f3c-9861-097d92dc2756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:12.438318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa40bf-47fd-4f3c-9861-097d92dc2756 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:12.447997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 154/310] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:12.453640 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb626c1-d5b2-490b-98f3-a83406cd46cc None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:12.561380 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb626c1-d5b2-490b-98f3-a83406cd46cc None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:12.574536 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa40bf-47fd-4f3c-9861-097d92dc2756 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:12.597705 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1c2c3-6d55-41cd-97c7-fbead0ed3622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:12.598404 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1c2c3-6d55-41cd-97c7-fbead0ed3622 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:12.598809 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1c2c3-6d55-41cd-97c7-fbead0ed3622 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:12.599264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1c2c3-6d55-41cd-97c7-fbead0ed3622 None admin] PATH_INFO: `/v3/users/6e46aede8b6347c088c36c65bdc84fff` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:12.600438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1c2c3-6d55-41cd-97c7-fbead0ed3622 None admin] RBAC: Authorizing `identity:get_user(user_id=6e46aede8b6347c088c36c65bdc84fff)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:12.601889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1c2c3-6d55-41cd-97c7-fbead0ed3622 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:12.603931 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 157/311] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:12 2022] GET /identity/v3/users/6e46aede8b6347c088c36c65bdc84fff => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:12.609480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1c2c3-6d55-41cd-97c7-fbead0ed3622 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:12.626489 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:12.696347 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa40bf-47fd-4f3c-9861-097d92dc2756 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:12.742600 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1c2c3-6d55-41cd-97c7-fbead0ed3622 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:12.743522 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774f43d-0bf8-4d49-81a1-0d4df4953317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:12.746405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774f43d-0bf8-4d49-81a1-0d4df4953317 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:12.747215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774f43d-0bf8-4d49-81a1-0d4df4953317 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:12.747952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774f43d-0bf8-4d49-81a1-0d4df4953317 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:12.765737 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774f43d-0bf8-4d49-81a1-0d4df4953317 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:12.774637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-0774f43d-0bf8-4d49-81a1-0d4df4953317 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:05:12.774637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:12.774637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:12.774637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:12.774637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:12.774637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:12.774637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:12.774637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:12.774637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:12.774637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:12.774637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:12.774637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:12.774637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:12.774637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:12.774637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:12.774637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:12.775729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:12.775729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:12.775729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:12.775729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:12.775729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:12.775729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:12.775729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:12.775729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:12.775729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:12.775729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:12.775729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:12.775729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:12.775729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:12.775729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:12.775729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:12.775729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:12.776762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:12.776762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:12.776762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:12.776762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:12.776762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:12.776762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:12.776762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:12.776762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:12.776762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:12.776762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:12.776762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:05:12.776762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:12.783883 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 155/312] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:12.785518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774f43d-0bf8-4d49-81a1-0d4df4953317 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:12.786988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ced565-b223-41a2-805c-cf285d68776c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:12.788225 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ced565-b223-41a2-805c-cf285d68776c None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:12.788710 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ced565-b223-41a2-805c-cf285d68776c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:12.789535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ced565-b223-41a2-805c-cf285d68776c None admin] PATH_INFO: `/v3/system/users/6e46aede8b6347c088c36c65bdc84fff/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:12.792322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ced565-b223-41a2-805c-cf285d68776c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6e46aede8b6347c088c36c65bdc84fff, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:12.793923 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ced565-b223-41a2-805c-cf285d68776c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:12.814938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:12.893479 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774f43d-0bf8-4d49-81a1-0d4df4953317 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:12.896664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 158/313] 10.222.0.59 () {58 vars in 1334 bytes} [Wed Aug 31 19:05:12 2022] PUT /identity/v3/system/users/6e46aede8b6347c088c36c65bdc84fff/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:05:12.948326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66796998-e419-4acc-a0f0-8d3be2f88688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:12.948871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66796998-e419-4acc-a0f0-8d3be2f88688 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:12.949154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66796998-e419-4acc-a0f0-8d3be2f88688 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:12.949483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66796998-e419-4acc-a0f0-8d3be2f88688 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:12.950173 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66796998-e419-4acc-a0f0-8d3be2f88688 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:12.950480 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66796998-e419-4acc-a0f0-8d3be2f88688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:12.952100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66796998-e419-4acc-a0f0-8d3be2f88688 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:12.963518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 156/314] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:12.969364 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ced565-b223-41a2-805c-cf285d68776c None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:13.063492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:13.095759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ced565-b223-41a2-805c-cf285d68776c None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:13.134412 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4d899-6b39-4c3f-a66d-cac6934326f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:13.135440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4d899-6b39-4c3f-a66d-cac6934326f2 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.135929 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4d899-6b39-4c3f-a66d-cac6934326f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.136431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4d899-6b39-4c3f-a66d-cac6934326f2 None admin] PATH_INFO: `/v3/users/24b84cadee624625b976c8cb24ee02b1` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.137781 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4d899-6b39-4c3f-a66d-cac6934326f2 None admin] RBAC: Authorizing `identity:get_user(user_id=24b84cadee624625b976c8cb24ee02b1)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:13.139528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4d899-6b39-4c3f-a66d-cac6934326f2 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:13.147880 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66796998-e419-4acc-a0f0-8d3be2f88688 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:13.148580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 159/315] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:12 2022] GET /identity/v3/users/24b84cadee624625b976c8cb24ee02b1 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:13.241489 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:13.258323 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66796998-e419-4acc-a0f0-8d3be2f88688 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:13.296291 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec826e-925f-4a8a-bf4a-04d4da12e882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:13.296912 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec826e-925f-4a8a-bf4a-04d4da12e882 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.297195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec826e-925f-4a8a-bf4a-04d4da12e882 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.297465 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec826e-925f-4a8a-bf4a-04d4da12e882 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.456637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec826e-925f-4a8a-bf4a-04d4da12e882 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:13.458697 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec826e-925f-4a8a-bf4a-04d4da12e882 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:13.554085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:13.572245 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 157/316] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:13.578047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4d899-6b39-4c3f-a66d-cac6934326f2 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:13.691006 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4d899-6b39-4c3f-a66d-cac6934326f2 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:13.728268 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef356a77-90b2-41b2-9309-9e2657b70152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:13.728903 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef356a77-90b2-41b2-9309-9e2657b70152 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.729180 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef356a77-90b2-41b2-9309-9e2657b70152 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.729458 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef356a77-90b2-41b2-9309-9e2657b70152 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.730565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef356a77-90b2-41b2-9309-9e2657b70152 None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:13.732053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef356a77-90b2-41b2-9309-9e2657b70152 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:13.734026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 160/317] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:13 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:13.740135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec826e-925f-4a8a-bf4a-04d4da12e882 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:13.847953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec826e-925f-4a8a-bf4a-04d4da12e882 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:13.884480 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99065b7-8b27-492a-8faa-be48e74088e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:13.885444 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99065b7-8b27-492a-8faa-be48e74088e1 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.885954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99065b7-8b27-492a-8faa-be48e74088e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.886428 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99065b7-8b27-492a-8faa-be48e74088e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.887425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99065b7-8b27-492a-8faa-be48e74088e1 None admin] RBAC: Adding query filter params (role.id=6c1ffb941f1646d2bf707005c7796790, scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d, user.id=24b84cadee624625b976c8cb24ee02b1) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:13.887941 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99065b7-8b27-492a-8faa-be48e74088e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:13.889609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99065b7-8b27-492a-8faa-be48e74088e1 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:13.896903 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 158/318] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:13 2022] GET /identity/v3/role_assignments?user.id=24b84cadee624625b976c8cb24ee02b1&scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d&role.id=6c1ffb941f1646d2bf707005c7796790 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:13.950447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f93f6-1a9a-4f90-8adf-b7ff289948e1 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.951021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f93f6-1a9a-4f90-8adf-b7ff289948e1 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.951021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f93f6-1a9a-4f90-8adf-b7ff289948e1 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.952801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 161/319] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:13.958788 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4aff7e-0284-4c24-88d0-fa943eeb466c None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.959121 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4aff7e-0284-4c24-88d0-fa943eeb466c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.959496 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4aff7e-0284-4c24-88d0-fa943eeb466c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.979777 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-2b4aff7e-0284-4c24-88d0-fa943eeb466c None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:13.981022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-2b4aff7e-0284-4c24-88d0-fa943eeb466c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:14.034003 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:14.064799 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 159/320] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:14.330530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a71a0-32df-4a41-a049-47e2cf25357d None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:14.330919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a71a0-32df-4a41-a049-47e2cf25357d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:14.331105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a71a0-32df-4a41-a049-47e2cf25357d None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:14.332577 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 162/321] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:14.337833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5fdcf-1183-45ef-ba54-fc2673efa8c8 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:14.338125 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5fdcf-1183-45ef-ba54-fc2673efa8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:14.338405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5fdcf-1183-45ef-ba54-fc2673efa8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:14.343218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a71a0-32df-4a41-a049-47e2cf25357d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:14.356696 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-3db5fdcf-1183-45ef-ba54-fc2673efa8c8 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:14.357879 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3db5fdcf-1183-45ef-ba54-fc2673efa8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:14.359213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:14.433335 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 160/322] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:14.454581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a71a0-32df-4a41-a049-47e2cf25357d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:14.489782 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cc312-b912-46a4-a3d2-994a4e3b0d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:14.490744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cc312-b912-46a4-a3d2-994a4e3b0d2a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:14.491098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cc312-b912-46a4-a3d2-994a4e3b0d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:14.491375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cc312-b912-46a4-a3d2-994a4e3b0d2a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:14.501867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09cc312-b912-46a4-a3d2-994a4e3b0d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:14.507584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-a09cc312-b912-46a4-a3d2-994a4e3b0d2a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:05:14.507584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:14.507584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:14.507584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:14.507584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:14.507584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:14.507584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:14.507584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:14.507584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:14.507584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:14.507584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:14.507584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:14.507584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:14.507584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:14.507584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:14.507584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:14.508519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:14.508519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:14.508519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:14.508519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:14.508519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:14.508519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:14.508519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:14.508519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:14.508519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:14.508519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:14.508519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:14.508519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:14.508519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:14.508519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:14.508519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:14.508519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:14.509328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:14.509328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:14.509328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:14.509328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:14.509328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:14.509328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:14.509328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:14.509328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:14.509328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:14.509328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:14.509328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:05:14.509328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:14.511180 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 163/323] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:14.514491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5fdcf-1183-45ef-ba54-fc2673efa8c8 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:14.620206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5fdcf-1183-45ef-ba54-fc2673efa8c8 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:14.630269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:14.653965 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eddd4c-4e32-4670-bca1-c61d315aa72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:14.655101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eddd4c-4e32-4670-bca1-c61d315aa72d None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:14.655771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eddd4c-4e32-4670-bca1-c61d315aa72d None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:14.656518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eddd4c-4e32-4670-bca1-c61d315aa72d None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:14.657189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eddd4c-4e32-4670-bca1-c61d315aa72d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:14.657493 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eddd4c-4e32-4670-bca1-c61d315aa72d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:14.658883 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eddd4c-4e32-4670-bca1-c61d315aa72d None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:14.670905 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 161/324] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:14.673762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cc312-b912-46a4-a3d2-994a4e3b0d2a None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:14.715129 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eddd4c-4e32-4670-bca1-c61d315aa72d None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:14.748675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:14.780130 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cc312-b912-46a4-a3d2-994a4e3b0d2a None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:14.818267 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ea563-2853-4640-a22b-b78b96e3f858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:14.819301 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ea563-2853-4640-a22b-b78b96e3f858 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:14.819761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ea563-2853-4640-a22b-b78b96e3f858 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:14.820260 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ea563-2853-4640-a22b-b78b96e3f858 None admin] PATH_INFO: `/v3/users/4c976853b632408a98e320c398c99f13` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:14.821565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ea563-2853-4640-a22b-b78b96e3f858 None admin] RBAC: Authorizing `identity:get_user(user_id=4c976853b632408a98e320c398c99f13)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:14.823231 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ea563-2853-4640-a22b-b78b96e3f858 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:14.829704 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 164/325] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:14 2022] GET /identity/v3/users/4c976853b632408a98e320c398c99f13 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:14.830761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ea563-2853-4640-a22b-b78b96e3f858 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:14.831042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eddd4c-4e32-4670-bca1-c61d315aa72d None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:14.870498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85894c7-6b67-4e1d-89c7-953929df9247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:14.871040 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85894c7-6b67-4e1d-89c7-953929df9247 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:14.871326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85894c7-6b67-4e1d-89c7-953929df9247 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:14.871629 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85894c7-6b67-4e1d-89c7-953929df9247 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:14.872761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85894c7-6b67-4e1d-89c7-953929df9247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:14.874288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85894c7-6b67-4e1d-89c7-953929df9247 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:14.876064 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 162/326] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:14.881289 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85894c7-6b67-4e1d-89c7-953929df9247 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:14.925316 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:14.945260 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ea563-2853-4640-a22b-b78b96e3f858 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:14.983767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef677c-54ad-41d0-9ee9-9e7593cadfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:14.984532 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef677c-54ad-41d0-9ee9-9e7593cadfb0 None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:14.984763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef677c-54ad-41d0-9ee9-9e7593cadfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:14.985047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef677c-54ad-41d0-9ee9-9e7593cadfb0 None admin] PATH_INFO: `/v3/system/users/4c976853b632408a98e320c398c99f13/roles/6c1ffb941f1646d2bf707005c7796790` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:14.986464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef677c-54ad-41d0-9ee9-9e7593cadfb0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4c976853b632408a98e320c398c99f13, role_id=6c1ffb941f1646d2bf707005c7796790)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:14.987874 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef677c-54ad-41d0-9ee9-9e7593cadfb0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:14.998617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85894c7-6b67-4e1d-89c7-953929df9247 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:15.031869 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950128a-2269-4c0d-9dd9-b00d192bc463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:15.032474 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950128a-2269-4c0d-9dd9-b00d192bc463 None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:15.032758 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950128a-2269-4c0d-9dd9-b00d192bc463 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:15.033058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950128a-2269-4c0d-9dd9-b00d192bc463 None admin] PATH_INFO: `/v3/groups` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:15.033683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9950128a-2269-4c0d-9dd9-b00d192bc463 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:15.035146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9950128a-2269-4c0d-9dd9-b00d192bc463 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:15.055350 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 165/327] 10.222.0.59 () {58 vars in 1334 bytes} [Wed Aug 31 19:05:14 2022] PUT /identity/v3/system/users/4c976853b632408a98e320c398c99f13/roles/6c1ffb941f1646d2bf707005c7796790 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:05:15.056749 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387986ff-1b43-47a8-867a-141e18909c62 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:15.057063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387986ff-1b43-47a8-867a-141e18909c62 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:15.057338 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387986ff-1b43-47a8-867a-141e18909c62 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:15.058580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 166/328] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:15.064548 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5997b-1238-4a37-b92c-fb7b64944e1b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:15.064832 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5997b-1238-4a37-b92c-fb7b64944e1b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:15.065154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5997b-1238-4a37-b92c-fb7b64944e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:15.073515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:15.086475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-94a5997b-1238-4a37-b92c-fb7b64944e1b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:15.087796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-94a5997b-1238-4a37-b92c-fb7b64944e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:15.112839 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 163/329] 10.222.0.59 () {60 vars in 1134 bytes} [Wed Aug 31 19:05:14 2022] POST /identity/v3/groups => generated 226 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:05:15.171902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 167/330] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:05:15.446730 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950128a-2269-4c0d-9dd9-b00d192bc463 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:15.563277 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:15.567580 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950128a-2269-4c0d-9dd9-b00d192bc463 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:15.605427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52109082-ac7c-4f8f-8bdb-7453c2dd1118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:15.606102 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52109082-ac7c-4f8f-8bdb-7453c2dd1118 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:15.606384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52109082-ac7c-4f8f-8bdb-7453c2dd1118 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:15.606651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52109082-ac7c-4f8f-8bdb-7453c2dd1118 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:15.618853 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52109082-ac7c-4f8f-8bdb-7453c2dd1118 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:15.624295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-52109082-ac7c-4f8f-8bdb-7453c2dd1118 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:15.624295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:15.624295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:15.624295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:15.624295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:15.624295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:15.624295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:15.624295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:15.624295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:15.624295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:15.624295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:15.624295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:15.624295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:15.624295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:15.624295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:15.624295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:15.625198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:15.625198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:15.625198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:15.625198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:15.625198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:15.625198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:15.625198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:15.625198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:15.625198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:15.625198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:15.625198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:15.625198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:15.625198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:15.625198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:15.625198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:15.625198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:15.626017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:15.626017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:15.626017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:15.626017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:15.626017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:15.626017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:15.626017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:15.626017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:15.626017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:15.626017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:15.626017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:15.626017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:15.627632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 164/331] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:15.631331 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5997b-1238-4a37-b92c-fb7b64944e1b None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:15.753462 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5997b-1238-4a37-b92c-fb7b64944e1b None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:15.799553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf2554-9030-4eb7-84aa-cbee604c983e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:15.800186 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf2554-9030-4eb7-84aa-cbee604c983e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:15.800876 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf2554-9030-4eb7-84aa-cbee604c983e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:15.800876 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf2554-9030-4eb7-84aa-cbee604c983e None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:15.801380 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf2554-9030-4eb7-84aa-cbee604c983e None admin] RBAC: Adding query filter params (name=member) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:15.802215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf2554-9030-4eb7-84aa-cbee604c983e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:15.806895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf2554-9030-4eb7-84aa-cbee604c983e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:15.806895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:15.813665 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 168/332] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:15.820608 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52109082-ac7c-4f8f-8bdb-7453c2dd1118 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:15.965147 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52109082-ac7c-4f8f-8bdb-7453c2dd1118 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:16.008268 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f4f48-7e6e-47a9-ab70-30af9304f7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:16.008836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f4f48-7e6e-47a9-ab70-30af9304f7cc None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:16.009116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f4f48-7e6e-47a9-ab70-30af9304f7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:16.009388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f4f48-7e6e-47a9-ab70-30af9304f7cc None admin] PATH_INFO: `/v3/users/6e46aede8b6347c088c36c65bdc84fff` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:16.010525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f4f48-7e6e-47a9-ab70-30af9304f7cc None admin] RBAC: Authorizing `identity:get_user(user_id=6e46aede8b6347c088c36c65bdc84fff)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:16.012004 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f4f48-7e6e-47a9-ab70-30af9304f7cc None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:16.013580 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 165/333] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:15 2022] GET /identity/v3/users/6e46aede8b6347c088c36c65bdc84fff => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:16.019978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf2554-9030-4eb7-84aa-cbee604c983e None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:16.138015 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf2554-9030-4eb7-84aa-cbee604c983e None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:16.173839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dea5c4-b75b-432e-bd06-561ae4f4f57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:16.174454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dea5c4-b75b-432e-bd06-561ae4f4f57d None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:16.174733 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dea5c4-b75b-432e-bd06-561ae4f4f57d None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:16.175113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dea5c4-b75b-432e-bd06-561ae4f4f57d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:16.175834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dea5c4-b75b-432e-bd06-561ae4f4f57d None admin] RBAC: Adding query filter params (role.id=357fdcf3417c45319b9d424a5ab4e9ca, user.id=6e46aede8b6347c088c36c65bdc84fff, scope.system=all) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:16.176140 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dea5c4-b75b-432e-bd06-561ae4f4f57d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:16.177618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dea5c4-b75b-432e-bd06-561ae4f4f57d None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:16.191349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 169/334] 10.222.0.59 () {56 vars in 1308 bytes} [Wed Aug 31 19:05:16 2022] GET /identity/v3/role_assignments?user.id=6e46aede8b6347c088c36c65bdc84fff&scope.system=all&role.id=357fdcf3417c45319b9d424a5ab4e9ca => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:17.331906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4b996-5bfb-4987-9b24-9efadc661d49 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.331906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4b996-5bfb-4987-9b24-9efadc661d49 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.331906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4b996-5bfb-4987-9b24-9efadc661d49 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.333854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 166/335] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:17.337992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fbd1f-e11c-4116-96f9-2281297b9fe0 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.338842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fbd1f-e11c-4116-96f9-2281297b9fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.339409 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fbd1f-e11c-4116-96f9-2281297b9fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.363465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-b04fbd1f-e11c-4116-96f9-2281297b9fe0 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:17.364997 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b04fbd1f-e11c-4116-96f9-2281297b9fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:17.430403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:17.449106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 170/336] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:17.823002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b463bd9-aeed-44ac-893a-55fa5d4d91bb None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.823376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b463bd9-aeed-44ac-893a-55fa5d4d91bb None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.823557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b463bd9-aeed-44ac-893a-55fa5d4d91bb None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.825331 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 167/337] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:17.840296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0776512-c209-4998-bfc4-15bef544da67 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.840581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0776512-c209-4998-bfc4-15bef544da67 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.840843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0776512-c209-4998-bfc4-15bef544da67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.861427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-b0776512-c209-4998-bfc4-15bef544da67 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:17.862613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b0776512-c209-4998-bfc4-15bef544da67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:17.940874 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 171/338] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:18.013898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b463bd9-aeed-44ac-893a-55fa5d4d91bb None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:18.038210 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012b799-7745-42d1-ab8d-ade92c7d0fd4 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.038523 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012b799-7745-42d1-ab8d-ade92c7d0fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.038758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012b799-7745-42d1-ab8d-ade92c7d0fd4 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.040884 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 172/339] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:18.047106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14de76e-8b31-49d0-8916-87595ffd2c76 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.047394 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14de76e-8b31-49d0-8916-87595ffd2c76 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.047824 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14de76e-8b31-49d0-8916-87595ffd2c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.052215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:18.084588 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-f14de76e-8b31-49d0-8916-87595ffd2c76 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:18.085834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f14de76e-8b31-49d0-8916-87595ffd2c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:18.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b463bd9-aeed-44ac-893a-55fa5d4d91bb None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:18.191079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 173/340] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:05:18.235610 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83d1f8-492e-4a17-b5dd-a17703c130e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:18.236272 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83d1f8-492e-4a17-b5dd-a17703c130e6 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.236552 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83d1f8-492e-4a17-b5dd-a17703c130e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.236827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83d1f8-492e-4a17-b5dd-a17703c130e6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.252450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83d1f8-492e-4a17-b5dd-a17703c130e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:18.264219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-ff83d1f8-492e-4a17-b5dd-a17703c130e6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:05:18.264219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:18.264219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:18.264219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:18.264219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:18.264219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:18.264219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:18.264219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:18.264219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:18.264219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:18.264219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:18.264219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:18.264219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:18.264219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:18.264219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:18.264219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:18.276749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:18.276749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:18.276749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:18.276749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:18.276749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:18.276749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:18.276749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:18.276749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:18.276749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:18.276749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:18.276749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:18.276749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:18.276749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:18.276749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:18.276749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:18.276749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:18.278101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:18.278101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:18.278101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:18.278101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:18.278101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:18.278101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:18.278101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:18.278101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:18.278101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:18.278101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:18.278101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:05:18.278101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:18.278101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 168/341] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:18.278788 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14de76e-8b31-49d0-8916-87595ffd2c76 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:18.351761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:18.352364 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83d1f8-492e-4a17-b5dd-a17703c130e6 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:18.448054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14de76e-8b31-49d0-8916-87595ffd2c76 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:18.492109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:18.515962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83d1f8-492e-4a17-b5dd-a17703c130e6 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:18.528349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622f76b-370c-4a86-bd67-50148ecddfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:18.528981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622f76b-370c-4a86-bd67-50148ecddfc4 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.529259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622f76b-370c-4a86-bd67-50148ecddfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.529532 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622f76b-370c-4a86-bd67-50148ecddfc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.530205 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622f76b-370c-4a86-bd67-50148ecddfc4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:18.530511 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622f76b-370c-4a86-bd67-50148ecddfc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:18.531921 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622f76b-370c-4a86-bd67-50148ecddfc4 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:18.551031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 174/342] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:18.553198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0918bbb-b41e-48e7-b06f-29d58cc3f13e None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.553480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0918bbb-b41e-48e7-b06f-29d58cc3f13e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.553751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0918bbb-b41e-48e7-b06f-29d58cc3f13e None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.555080 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 175/343] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:18.558597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0918bbb-b41e-48e7-b06f-29d58cc3f13e None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:18.585627 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4b8ff-2ad2-4959-8b4f-95da3eeca5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:18.586288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4b8ff-2ad2-4959-8b4f-95da3eeca5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.586616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4b8ff-2ad2-4959-8b4f-95da3eeca5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.586912 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4b8ff-2ad2-4959-8b4f-95da3eeca5e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.611761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4b8ff-2ad2-4959-8b4f-95da3eeca5e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:18.618513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-9cf4b8ff-2ad2-4959-8b4f-95da3eeca5e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:18.618513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:18.618513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:18.618513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:18.618513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:18.618513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:18.618513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:18.618513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:18.618513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:18.618513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:18.618513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:18.618513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:18.618513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:18.618513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:18.618513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:18.618513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:18.619498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:18.619498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:18.619498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:18.619498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:18.619498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:18.619498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:18.619498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:18.619498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:18.619498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:18.619498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:18.619498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:18.619498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:18.619498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:18.619498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:18.619498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:18.619498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:18.620307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:18.620307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:18.620307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:18.620307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:18.620307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:18.620307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:18.620307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:18.620307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:18.620307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:18.620307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:18.620307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:18.620307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:18.626671 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 169/344] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:18.628969 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bb5cb-bb5e-49c1-842d-a9cdce4364fb None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.629259 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bb5cb-bb5e-49c1-842d-a9cdce4364fb None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.631714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bb5cb-bb5e-49c1-842d-a9cdce4364fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.670232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-394bb5cb-bb5e-49c1-842d-a9cdce4364fb None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:18.671287 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-394bb5cb-bb5e-49c1-842d-a9cdce4364fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:18.726741 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0918bbb-b41e-48e7-b06f-29d58cc3f13e None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:18.736877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:18.771958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae16f70-37f2-44a9-838b-11474f29c3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:18.772902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae16f70-37f2-44a9-838b-11474f29c3fd None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.773365 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae16f70-37f2-44a9-838b-11474f29c3fd None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.773851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae16f70-37f2-44a9-838b-11474f29c3fd None admin] PATH_INFO: `/v3/users/4c976853b632408a98e320c398c99f13` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.775275 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae16f70-37f2-44a9-838b-11474f29c3fd None admin] RBAC: Authorizing `identity:get_user(user_id=4c976853b632408a98e320c398c99f13)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:18.776920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae16f70-37f2-44a9-838b-11474f29c3fd None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:18.780678 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 170/345] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:18.785457 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 176/346] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:18 2022] GET /identity/v3/users/4c976853b632408a98e320c398c99f13 => generated 293 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:18.785580 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bb5cb-bb5e-49c1-842d-a9cdce4364fb None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:18.785673 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae16f70-37f2-44a9-838b-11474f29c3fd None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:18.878459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:18.902270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae16f70-37f2-44a9-838b-11474f29c3fd None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:18.905925 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bb5cb-bb5e-49c1-842d-a9cdce4364fb None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:18.942974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e060ce-d790-40ab-8cb7-2d6927f1b136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:18.944016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e060ce-d790-40ab-8cb7-2d6927f1b136 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.944498 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e060ce-d790-40ab-8cb7-2d6927f1b136 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.944761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66310c-db24-43f5-8289-b859a0979c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:18.945197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e060ce-d790-40ab-8cb7-2d6927f1b136 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.945418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66310c-db24-43f5-8289-b859a0979c8b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.945700 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66310c-db24-43f5-8289-b859a0979c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.945971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66310c-db24-43f5-8289-b859a0979c8b None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.946815 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66310c-db24-43f5-8289-b859a0979c8b None admin] RBAC: Adding query filter params (name=member) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:18.947165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66310c-db24-43f5-8289-b859a0979c8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:18.948593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66310c-db24-43f5-8289-b859a0979c8b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:18.957933 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 171/347] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:18.959314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66310c-db24-43f5-8289-b859a0979c8b None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:18.961159 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e060ce-d790-40ab-8cb7-2d6927f1b136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:18.970994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-55e060ce-d790-40ab-8cb7-2d6927f1b136 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:18.970994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:18.970994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:18.970994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:18.970994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:18.970994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:18.970994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:18.970994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:18.970994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:18.970994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:18.970994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:18.970994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:18.970994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:18.970994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:18.970994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:18.970994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:18.972054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:18.972054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:18.972054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:18.972054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:18.972054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:18.972054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:18.972054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:18.972054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:18.972054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:18.972054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:18.972054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:18.972054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:18.972054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:18.972054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:18.972054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:18.972054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:18.972898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:18.972898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:18.972898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:18.972898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:18.972898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:18.972898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:18.972898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:18.972898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:18.972898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:18.972898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:18.972898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:18.972898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:18.975444 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 177/348] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:18.981289 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e060ce-d790-40ab-8cb7-2d6927f1b136 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:19.019610 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:19.067976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66310c-db24-43f5-8289-b859a0979c8b None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:19.091412 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e060ce-d790-40ab-8cb7-2d6927f1b136 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:19.107005 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a84061-dd79-40ba-a0c1-9e57bfe18b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:19.107640 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a84061-dd79-40ba-a0c1-9e57bfe18b84 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:19.108224 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a84061-dd79-40ba-a0c1-9e57bfe18b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:19.108224 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a84061-dd79-40ba-a0c1-9e57bfe18b84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:19.108974 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a84061-dd79-40ba-a0c1-9e57bfe18b84 None admin] RBAC: Adding query filter params (role.id=6c1ffb941f1646d2bf707005c7796790, user.id=4c976853b632408a98e320c398c99f13, scope.system=all) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:19.109285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a84061-dd79-40ba-a0c1-9e57bfe18b84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:19.111078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a84061-dd79-40ba-a0c1-9e57bfe18b84 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:19.117566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:19.127231 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 172/349] 10.222.0.59 () {56 vars in 1308 bytes} [Wed Aug 31 19:05:18 2022] GET /identity/v3/role_assignments?user.id=4c976853b632408a98e320c398c99f13&scope.system=all&role.id=6c1ffb941f1646d2bf707005c7796790 => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:19.140378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a84061-dd79-40ba-a0c1-9e57bfe18b84 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:19.140521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef26016-bf24-438a-9cdd-28c8f4dc7e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:19.140521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef26016-bf24-438a-9cdd-28c8f4dc7e01 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:19.140521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef26016-bf24-438a-9cdd-28c8f4dc7e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:19.140521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef26016-bf24-438a-9cdd-28c8f4dc7e01 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:19.140799 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef26016-bf24-438a-9cdd-28c8f4dc7e01 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:19.140799 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef26016-bf24-438a-9cdd-28c8f4dc7e01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:19.140799 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef26016-bf24-438a-9cdd-28c8f4dc7e01 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:19.147434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 178/350] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:19.153688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef26016-bf24-438a-9cdd-28c8f4dc7e01 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:19.251767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a84061-dd79-40ba-a0c1-9e57bfe18b84 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:19.292836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff2071-d18a-4284-9c36-e8b3f8028a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:19.293461 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff2071-d18a-4284-9c36-e8b3f8028a2b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:19.293738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff2071-d18a-4284-9c36-e8b3f8028a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:19.294008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff2071-d18a-4284-9c36-e8b3f8028a2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:19.295636 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:19.306050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef26016-bf24-438a-9cdd-28c8f4dc7e01 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:19.369741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:19.372688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e19980-93ff-413b-ba45-154a7596f46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:19.373327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e19980-93ff-413b-ba45-154a7596f46a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:19.373613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e19980-93ff-413b-ba45-154a7596f46a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:19.373893 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e19980-93ff-413b-ba45-154a7596f46a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:19.377848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff2071-d18a-4284-9c36-e8b3f8028a2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:19.379619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff2071-d18a-4284-9c36-e8b3f8028a2b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:19.464559 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e19980-93ff-413b-ba45-154a7596f46a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:19.466083 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e19980-93ff-413b-ba45-154a7596f46a None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:19.493645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 173/351] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:19.495034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff2071-d18a-4284-9c36-e8b3f8028a2b None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:19.601568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:19.610910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 179/352] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 19:05:19.612295 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf249345-a1ea-4c15-bbb6-5d23401bba0d None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:19.614600 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf249345-a1ea-4c15-bbb6-5d23401bba0d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:19.614600 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf249345-a1ea-4c15-bbb6-5d23401bba0d None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:19.614946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 180/353] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:19.618283 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf249345-a1ea-4c15-bbb6-5d23401bba0d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:19.628772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:19.635294 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff2071-d18a-4284-9c36-e8b3f8028a2b None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:19.684001 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0250d-e322-4de4-bc29-e4c74fd13436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:19.684643 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0250d-e322-4de4-bc29-e4c74fd13436 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:19.684927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0250d-e322-4de4-bc29-e4c74fd13436 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:19.685199 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0250d-e322-4de4-bc29-e4c74fd13436 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:19.700540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0250d-e322-4de4-bc29-e4c74fd13436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:19.705410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-51c0250d-e322-4de4-bc29-e4c74fd13436 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:19.705410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:19.705410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:19.705410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:19.705410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:19.705410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:19.705410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:19.705410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:19.705410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:19.705410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:19.705410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:19.705410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:19.705410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:19.705410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:19.705410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:19.705410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:19.706488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:19.706488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:19.706488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:19.706488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:19.706488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:19.706488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:19.706488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:19.706488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:19.706488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:19.706488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:19.706488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:19.706488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:19.706488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:19.706488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:19.706488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:19.706488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:19.707481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:19.707481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:19.707481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:19.707481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:19.707481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:19.707481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:19.707481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:19.707481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:19.707481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:19.707481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:19.707481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:19.707481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:19.707481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 174/354] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:19.709101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0614f-e8c6-4463-8a29-d78cbfb5ef44 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:19.709270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0614f-e8c6-4463-8a29-d78cbfb5ef44 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:19.709579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0614f-e8c6-4463-8a29-d78cbfb5ef44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:19.730036 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-12d0614f-e8c6-4463-8a29-d78cbfb5ef44 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:19.731689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-12d0614f-e8c6-4463-8a29-d78cbfb5ef44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:19.767730 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf249345-a1ea-4c15-bbb6-5d23401bba0d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:19.810426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d348b3-27e2-472c-aed4-5dd7e4474643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:19.811493 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d348b3-27e2-472c-aed4-5dd7e4474643 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:19.812046 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d348b3-27e2-472c-aed4-5dd7e4474643 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:19.813503 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d348b3-27e2-472c-aed4-5dd7e4474643 None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:19.815027 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d348b3-27e2-472c-aed4-5dd7e4474643 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:19.817503 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d348b3-27e2-472c-aed4-5dd7e4474643 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:19.819057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 175/355] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:19.819434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 181/356] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:19 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:19.819798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0614f-e8c6-4463-8a29-d78cbfb5ef44 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:19.820080 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d348b3-27e2-472c-aed4-5dd7e4474643 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:19.897455 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:19.917975 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:19.933523 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0614f-e8c6-4463-8a29-d78cbfb5ef44 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:19.937175 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d348b3-27e2-472c-aed4-5dd7e4474643 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:19.971411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc262f5f-3fd7-4c5c-af08-f16827b80937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:19.972545 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc262f5f-3fd7-4c5c-af08-f16827b80937 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:19.972828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc262f5f-3fd7-4c5c-af08-f16827b80937 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:19.973103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc262f5f-3fd7-4c5c-af08-f16827b80937 None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:19.974711 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc262f5f-3fd7-4c5c-af08-f16827b80937 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:19.976449 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc262f5f-3fd7-4c5c-af08-f16827b80937 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:19.978411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 176/357] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:19 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:19.978411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc262f5f-3fd7-4c5c-af08-f16827b80937 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:19.982102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c9833-e970-484e-ac32-f13e9d138c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:19.982996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c9833-e970-484e-ac32-f13e9d138c2e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:19.983555 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c9833-e970-484e-ac32-f13e9d138c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:19.984038 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c9833-e970-484e-ac32-f13e9d138c2e None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:19.984907 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c9833-e970-484e-ac32-f13e9d138c2e None admin] RBAC: Adding query filter params (name=member) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:19.985445 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c9833-e970-484e-ac32-f13e9d138c2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:19.987055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c9833-e970-484e-ac32-f13e9d138c2e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:19.997595 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 182/358] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:19.998726 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c9833-e970-484e-ac32-f13e9d138c2e None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:20.100083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc262f5f-3fd7-4c5c-af08-f16827b80937 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:20.108769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c9833-e970-484e-ac32-f13e9d138c2e None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:20.139554 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861302f-492e-4910-80e6-09993e3c2b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:20.140203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861302f-492e-4910-80e6-09993e3c2b5c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:20.140488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861302f-492e-4910-80e6-09993e3c2b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:20.140744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861302f-492e-4910-80e6-09993e3c2b5c None admin] PATH_INFO: `/v3/groups/a96ee9db2ec744d288e355aa008e72a8` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:20.143731 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:20.149173 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861302f-492e-4910-80e6-09993e3c2b5c None admin] RBAC: Authorizing `identity:get_group(group_id=a96ee9db2ec744d288e355aa008e72a8)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:20.150588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861302f-492e-4910-80e6-09993e3c2b5c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:20.152371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 177/359] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 31 19:05:19 2022] GET /identity/v3/groups/a96ee9db2ec744d288e355aa008e72a8 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:20.153453 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861302f-492e-4910-80e6-09993e3c2b5c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:20.159075 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a606712-6402-43c6-a9f7-fe22eb53ca4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:20.159807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a606712-6402-43c6-a9f7-fe22eb53ca4c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:20.160159 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a606712-6402-43c6-a9f7-fe22eb53ca4c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:20.160498 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a606712-6402-43c6-a9f7-fe22eb53ca4c None admin] PATH_INFO: `/v3/groups/a96ee9db2ec744d288e355aa008e72a8` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:20.162126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a606712-6402-43c6-a9f7-fe22eb53ca4c None admin] RBAC: Authorizing `identity:get_group(group_id=a96ee9db2ec744d288e355aa008e72a8)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:20.164131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a606712-6402-43c6-a9f7-fe22eb53ca4c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:20.165825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 183/360] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 31 19:05:19 2022] GET /identity/v3/groups/a96ee9db2ec744d288e355aa008e72a8 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:20.167666 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a606712-6402-43c6-a9f7-fe22eb53ca4c None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:20.188858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:20.300427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861302f-492e-4910-80e6-09993e3c2b5c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:20.321241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a606712-6402-43c6-a9f7-fe22eb53ca4c None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:20.344498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d28f56-dc9c-45eb-96fb-7c972dd1d889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:20.345122 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d28f56-dc9c-45eb-96fb-7c972dd1d889 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:20.345403 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d28f56-dc9c-45eb-96fb-7c972dd1d889 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:20.345674 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d28f56-dc9c-45eb-96fb-7c972dd1d889 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:20.371700 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c59126-cf30-4543-bcfd-2de4a374dd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:20.372328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59126-cf30-4543-bcfd-2de4a374dd92 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:20.372605 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59126-cf30-4543-bcfd-2de4a374dd92 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:20.373033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59126-cf30-4543-bcfd-2de4a374dd92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:20.374102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c59126-cf30-4543-bcfd-2de4a374dd92 None admin] RBAC: Adding query filter params (group.id=a96ee9db2ec744d288e355aa008e72a8, role.id=357fdcf3417c45319b9d424a5ab4e9ca, scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:20.374412 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c59126-cf30-4543-bcfd-2de4a374dd92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:20.375871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c59126-cf30-4543-bcfd-2de4a374dd92 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:20.383420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 184/361] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 31 19:05:20 2022] GET /identity/v3/role_assignments?group.id=a96ee9db2ec744d288e355aa008e72a8&scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69&role.id=357fdcf3417c45319b9d424a5ab4e9ca => generated 260 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:20.384681 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c59126-cf30-4543-bcfd-2de4a374dd92 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:20.433719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d28f56-dc9c-45eb-96fb-7c972dd1d889 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:20.436386 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d28f56-dc9c-45eb-96fb-7c972dd1d889 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:20.462001 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:20.471841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:20.534997 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c59126-cf30-4543-bcfd-2de4a374dd92 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:20.581002 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e820a-f2a0-48d5-9413-39916d6f4c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:20.581655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e820a-f2a0-48d5-9413-39916d6f4c9e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:20.581933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e820a-f2a0-48d5-9413-39916d6f4c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:20.582208 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e820a-f2a0-48d5-9413-39916d6f4c9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:20.587524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 178/362] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:20.588511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d28f56-dc9c-45eb-96fb-7c972dd1d889 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:20.594671 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e820a-f2a0-48d5-9413-39916d6f4c9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:20.601713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-c14e820a-f2a0-48d5-9413-39916d6f4c9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:20.601713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:20.601713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:20.601713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:20.601713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:20.601713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:20.601713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:20.601713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:20.601713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:20.601713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:20.601713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:20.601713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:20.601713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:20.601713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:20.601713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:20.601713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:20.602680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:20.602680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:20.602680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:20.602680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:20.602680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:20.602680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:20.602680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:20.602680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:20.602680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:20.602680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:20.602680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:20.602680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:20.602680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:20.602680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:20.602680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:20.602680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:20.603785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:20.603785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:20.603785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:20.603785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:20.603785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:20.603785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:20.603785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:20.603785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:20.603785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:20.603785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:20.603785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:20.603785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:20.603785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 185/363] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:20.608686 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85e4c8-d1f6-4681-9ee4-adbc4628c7a3 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:20.608966 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85e4c8-d1f6-4681-9ee4-adbc4628c7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:20.609235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85e4c8-d1f6-4681-9ee4-adbc4628c7a3 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:20.610794 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 186/364] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:20.611254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85e4c8-d1f6-4681-9ee4-adbc4628c7a3 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:20.744583 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d28f56-dc9c-45eb-96fb-7c972dd1d889 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:20.766949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:20.768100 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85e4c8-d1f6-4681-9ee4-adbc4628c7a3 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:20.789016 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91986fd7-aa7a-4a18-83e3-bbd84991012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:20.789568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91986fd7-aa7a-4a18-83e3-bbd84991012c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:20.789850 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91986fd7-aa7a-4a18-83e3-bbd84991012c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:20.790113 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91986fd7-aa7a-4a18-83e3-bbd84991012c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:20.790850 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91986fd7-aa7a-4a18-83e3-bbd84991012c None admin] RBAC: Adding query filter params (group.id=a96ee9db2ec744d288e355aa008e72a8, role.id=e14f37f4a3a24f8da0f757aabb1f39be, scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:20.791192 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91986fd7-aa7a-4a18-83e3-bbd84991012c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:20.791400 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:20.792628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91986fd7-aa7a-4a18-83e3-bbd84991012c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:20.800300 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 179/365] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 31 19:05:20 2022] GET /identity/v3/role_assignments?group.id=a96ee9db2ec744d288e355aa008e72a8&scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69&role.id=e14f37f4a3a24f8da0f757aabb1f39be => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:20.801241 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91986fd7-aa7a-4a18-83e3-bbd84991012c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:20.812806 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38062f3-46d8-42bd-8037-c6a7ef6383c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:20.820800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38062f3-46d8-42bd-8037-c6a7ef6383c0 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:20.820800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38062f3-46d8-42bd-8037-c6a7ef6383c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:20.820800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38062f3-46d8-42bd-8037-c6a7ef6383c0 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:20.820800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38062f3-46d8-42bd-8037-c6a7ef6383c0 None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:20.820800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38062f3-46d8-42bd-8037-c6a7ef6383c0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:20.822445 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 187/366] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:20 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:20.828678 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69058769-af18-400b-acc4-765a4ee8abe9 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:20.829404 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69058769-af18-400b-acc4-765a4ee8abe9 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:20.829404 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69058769-af18-400b-acc4-765a4ee8abe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:20.853563 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-69058769-af18-400b-acc4-765a4ee8abe9 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:20.854982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-69058769-af18-400b-acc4-765a4ee8abe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:20.934523 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91986fd7-aa7a-4a18-83e3-bbd84991012c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:20.941255 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 188/367] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:20.942149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69058769-af18-400b-acc4-765a4ee8abe9 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:20.974802 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30642bba-f6cc-474a-a684-2070488c14f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:20.975794 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30642bba-f6cc-474a-a684-2070488c14f2 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:20.976461 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30642bba-f6cc-474a-a684-2070488c14f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:20.977207 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30642bba-f6cc-474a-a684-2070488c14f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:20.978366 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30642bba-f6cc-474a-a684-2070488c14f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:20.978917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30642bba-f6cc-474a-a684-2070488c14f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:20.980587 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30642bba-f6cc-474a-a684-2070488c14f2 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:20.990011 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 180/368] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:20.995678 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30642bba-f6cc-474a-a684-2070488c14f2 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:21.052935 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:21.121369 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30642bba-f6cc-474a-a684-2070488c14f2 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:21.162960 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69058769-af18-400b-acc4-765a4ee8abe9 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:21.164377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184a714-d9f2-4f5f-955c-b1862ef44e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:21.165468 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184a714-d9f2-4f5f-955c-b1862ef44e2d None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.165987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184a714-d9f2-4f5f-955c-b1862ef44e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.166487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184a714-d9f2-4f5f-955c-b1862ef44e2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.191173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:21.225832 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01138f6c-10bb-4842-8119-38a6cfec6a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:21.226373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01138f6c-10bb-4842-8119-38a6cfec6a5f None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.226653 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01138f6c-10bb-4842-8119-38a6cfec6a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.226978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01138f6c-10bb-4842-8119-38a6cfec6a5f None admin] PATH_INFO: `/v3/groups/a96ee9db2ec744d288e355aa008e72a8` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.228091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01138f6c-10bb-4842-8119-38a6cfec6a5f None admin] RBAC: Authorizing `identity:get_group(group_id=a96ee9db2ec744d288e355aa008e72a8)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:21.229648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01138f6c-10bb-4842-8119-38a6cfec6a5f None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:21.235957 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 189/369] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 31 19:05:20 2022] GET /identity/v3/groups/a96ee9db2ec744d288e355aa008e72a8 => generated 226 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:21.241286 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01138f6c-10bb-4842-8119-38a6cfec6a5f None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:21.256690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184a714-d9f2-4f5f-955c-b1862ef44e2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:21.258470 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184a714-d9f2-4f5f-955c-b1862ef44e2d None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:21.316738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:21.396580 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 181/370] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:21.405762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184a714-d9f2-4f5f-955c-b1862ef44e2d None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:21.463251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01138f6c-10bb-4842-8119-38a6cfec6a5f None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:21.503565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186acc1f-0021-4ecb-a87c-17cf63964240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:21.504184 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186acc1f-0021-4ecb-a87c-17cf63964240 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.505040 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186acc1f-0021-4ecb-a87c-17cf63964240 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.505312 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186acc1f-0021-4ecb-a87c-17cf63964240 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.506292 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186acc1f-0021-4ecb-a87c-17cf63964240 None admin] RBAC: Adding query filter params (group.id=a96ee9db2ec744d288e355aa008e72a8, role.id=357fdcf3417c45319b9d424a5ab4e9ca, scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:21.506611 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186acc1f-0021-4ecb-a87c-17cf63964240 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:21.508105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186acc1f-0021-4ecb-a87c-17cf63964240 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:21.521825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 190/371] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 31 19:05:21 2022] GET /identity/v3/role_assignments?group.id=a96ee9db2ec744d288e355aa008e72a8&scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d&role.id=357fdcf3417c45319b9d424a5ab4e9ca => generated 260 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:21.522950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186acc1f-0021-4ecb-a87c-17cf63964240 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:21.546346 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184a714-d9f2-4f5f-955c-b1862ef44e2d None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:21.600097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d564e5-1f6e-4024-bc17-cfadbd4f8aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:21.600865 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d564e5-1f6e-4024-bc17-cfadbd4f8aeb None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.602217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d564e5-1f6e-4024-bc17-cfadbd4f8aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.602217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d564e5-1f6e-4024-bc17-cfadbd4f8aeb None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.603208 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d564e5-1f6e-4024-bc17-cfadbd4f8aeb None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:21.605946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d564e5-1f6e-4024-bc17-cfadbd4f8aeb None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:21.608257 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 182/372] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:21 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:21.616060 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d564e5-1f6e-4024-bc17-cfadbd4f8aeb None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:21.632270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:21.637365 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:21.724922 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186acc1f-0021-4ecb-a87c-17cf63964240 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:21.764356 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d564e5-1f6e-4024-bc17-cfadbd4f8aeb None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:21.765262 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709f4d7-a1ec-45a2-a676-21e272a3e2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:21.766010 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709f4d7-a1ec-45a2-a676-21e272a3e2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.766915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709f4d7-a1ec-45a2-a676-21e272a3e2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.766915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709f4d7-a1ec-45a2-a676-21e272a3e2c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.779103 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709f4d7-a1ec-45a2-a676-21e272a3e2c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:21.785688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-a709f4d7-a1ec-45a2-a676-21e272a3e2c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:21.785688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:21.785688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:21.785688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:21.785688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:21.785688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:21.785688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:21.785688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:21.785688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:21.785688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:21.785688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:21.785688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:21.785688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:21.785688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:21.785688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:21.785688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:21.787308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:21.787308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:21.787308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:21.787308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:21.787308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:21.787308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:21.787308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:21.787308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:21.787308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:21.787308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:21.787308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:21.787308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:21.787308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:21.787308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:21.787308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:21.787308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:21.788131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:21.788131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:21.788131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:21.788131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:21.788131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:21.788131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:21.788131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:21.788131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:21.788131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:21.788131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:21.788131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:21.788131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:21.788131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 191/373] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:05:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:21.793731 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709f4d7-a1ec-45a2-a676-21e272a3e2c1 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:21.839080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88ad83-f49d-46f9-b0bc-5607dc9048b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:21.839730 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88ad83-f49d-46f9-b0bc-5607dc9048b6 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.840007 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88ad83-f49d-46f9-b0bc-5607dc9048b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.840280 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88ad83-f49d-46f9-b0bc-5607dc9048b6 None admin] PATH_INFO: `/v3/groups/a96ee9db2ec744d288e355aa008e72a8` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.841508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e88ad83-f49d-46f9-b0bc-5607dc9048b6 None admin] RBAC: Authorizing `identity:get_group(group_id=a96ee9db2ec744d288e355aa008e72a8)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:21.842974 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e88ad83-f49d-46f9-b0bc-5607dc9048b6 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:21.845179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 183/374] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 31 19:05:21 2022] GET /identity/v3/groups/a96ee9db2ec744d288e355aa008e72a8 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:21.851545 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88ad83-f49d-46f9-b0bc-5607dc9048b6 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:21.909947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709f4d7-a1ec-45a2-a676-21e272a3e2c1 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:21.921188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:21.956762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54915418-143c-4116-a94c-2ccb3544f296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:21.957440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54915418-143c-4116-a94c-2ccb3544f296 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.958277 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54915418-143c-4116-a94c-2ccb3544f296 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.958563 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54915418-143c-4116-a94c-2ccb3544f296 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.960273 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54915418-143c-4116-a94c-2ccb3544f296 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:21.960586 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54915418-143c-4116-a94c-2ccb3544f296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:21.961985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54915418-143c-4116-a94c-2ccb3544f296 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:21.968536 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:21.986713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 192/375] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:22.000577 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88ad83-f49d-46f9-b0bc-5607dc9048b6 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:22.005274 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54915418-143c-4116-a94c-2ccb3544f296 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:22.064453 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c57c2a-c579-4503-bd69-3888494e6c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:22.065228 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c57c2a-c579-4503-bd69-3888494e6c58 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:22.065632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c57c2a-c579-4503-bd69-3888494e6c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:22.065921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c57c2a-c579-4503-bd69-3888494e6c58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:22.066664 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c57c2a-c579-4503-bd69-3888494e6c58 None admin] RBAC: Adding query filter params (group.id=a96ee9db2ec744d288e355aa008e72a8, role.id=e14f37f4a3a24f8da0f757aabb1f39be, scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:22.068768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c57c2a-c579-4503-bd69-3888494e6c58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:22.070743 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c57c2a-c579-4503-bd69-3888494e6c58 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:22.077936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 184/376] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 31 19:05:21 2022] GET /identity/v3/role_assignments?group.id=a96ee9db2ec744d288e355aa008e72a8&scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d&role.id=e14f37f4a3a24f8da0f757aabb1f39be => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:22.162826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54915418-143c-4116-a94c-2ccb3544f296 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:22.204161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4b39b-b037-492a-9de3-ad129a63f011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:22.204791 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4b39b-b037-492a-9de3-ad129a63f011 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:22.205071 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4b39b-b037-492a-9de3-ad129a63f011 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:22.205345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4b39b-b037-492a-9de3-ad129a63f011 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:22.251245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:22.283875 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4b39b-b037-492a-9de3-ad129a63f011 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:22.285349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4b39b-b037-492a-9de3-ad129a63f011 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:22.397042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 193/377] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:22.402762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c57c2a-c579-4503-bd69-3888494e6c58 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:22.519446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c57c2a-c579-4503-bd69-3888494e6c58 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:22.556964 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba0b35-4010-4846-a88c-b92aaf824e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:22.557584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba0b35-4010-4846-a88c-b92aaf824e28 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:22.557867 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba0b35-4010-4846-a88c-b92aaf824e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:22.558141 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba0b35-4010-4846-a88c-b92aaf824e28 None admin] PATH_INFO: `/v3/projects/c9e5945a1a4d41fca11373bc396c02fd` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:22.559307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba0b35-4010-4846-a88c-b92aaf824e28 None admin] RBAC: Authorizing `identity:get_project(project_id=c9e5945a1a4d41fca11373bc396c02fd)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:22.560741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba0b35-4010-4846-a88c-b92aaf824e28 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:22.562681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 185/378] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:22 2022] GET /identity/v3/projects/c9e5945a1a4d41fca11373bc396c02fd => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:22.572004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4b39b-b037-492a-9de3-ad129a63f011 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:22.668952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8985e-e8d7-4544-abd7-58921bbf9588 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:22.669592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8985e-e8d7-4544-abd7-58921bbf9588 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:22.670077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8985e-e8d7-4544-abd7-58921bbf9588 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:22.671661 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 186/379] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:22.675747 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:22.701296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ceb602-cee9-403a-a3c5-1b9675475672 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:22.701787 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ceb602-cee9-403a-a3c5-1b9675475672 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:22.702282 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ceb602-cee9-403a-a3c5-1b9675475672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:22.719357 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4b39b-b037-492a-9de3-ad129a63f011 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:22.726285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-e0ceb602-cee9-403a-a3c5-1b9675475672 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:22.728668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e0ceb602-cee9-403a-a3c5-1b9675475672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:22.747242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:22.793745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c3560-711f-49bf-bf40-8fe1ed513831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:22.794367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c3560-711f-49bf-bf40-8fe1ed513831 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:22.794652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c3560-711f-49bf-bf40-8fe1ed513831 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:22.794956 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c3560-711f-49bf-bf40-8fe1ed513831 None admin] PATH_INFO: `/v3/groups/e2a2d4a89fe640229d7c3f331570ad1f` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:22.816155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c3560-711f-49bf-bf40-8fe1ed513831 None admin] RBAC: Authorizing `identity:get_group(group_id=e2a2d4a89fe640229d7c3f331570ad1f)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:22.818453 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c3560-711f-49bf-bf40-8fe1ed513831 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:22.820850 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 194/380] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 31 19:05:22 2022] GET /identity/v3/groups/e2a2d4a89fe640229d7c3f331570ad1f => generated 229 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:22.822068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 187/381] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:22.836244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c3560-711f-49bf-bf40-8fe1ed513831 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:22.995359 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c3560-711f-49bf-bf40-8fe1ed513831 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:23.046233 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63a02b-bd6e-44fe-9de3-edde58bab478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:23.047121 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63a02b-bd6e-44fe-9de3-edde58bab478 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:23.047547 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63a02b-bd6e-44fe-9de3-edde58bab478 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:23.047962 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63a02b-bd6e-44fe-9de3-edde58bab478 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:23.048851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63a02b-bd6e-44fe-9de3-edde58bab478 None admin] RBAC: Adding query filter params (group.id=e2a2d4a89fe640229d7c3f331570ad1f, role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=c9e5945a1a4d41fca11373bc396c02fd) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:23.049438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63a02b-bd6e-44fe-9de3-edde58bab478 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:23.051045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63a02b-bd6e-44fe-9de3-edde58bab478 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:23.054056 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:23.059559 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 195/382] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 31 19:05:22 2022] GET /identity/v3/role_assignments?group.id=e2a2d4a89fe640229d7c3f331570ad1f&scope.project.id=c9e5945a1a4d41fca11373bc396c02fd&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:23.151997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1aab2-c012-432a-8d6f-3fde599686c3 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:23.152279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1aab2-c012-432a-8d6f-3fde599686c3 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:23.152556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1aab2-c012-432a-8d6f-3fde599686c3 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:23.154192 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 188/383] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:23.165129 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89122507-bd60-4657-8f80-d490eb00ceb2 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:23.165758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89122507-bd60-4657-8f80-d490eb00ceb2 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:23.166235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89122507-bd60-4657-8f80-d490eb00ceb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:23.192064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-89122507-bd60-4657-8f80-d490eb00ceb2 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:23.193582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-89122507-bd60-4657-8f80-d490eb00ceb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:23.284305 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 196/384] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:23.335828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1aab2-c012-432a-8d6f-3fde599686c3 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:23.459183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1aab2-c012-432a-8d6f-3fde599686c3 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:23.503144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32412a7-6424-4412-a531-5b9985a01e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:23.503845 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32412a7-6424-4412-a531-5b9985a01e11 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:23.504131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32412a7-6424-4412-a531-5b9985a01e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:23.504984 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32412a7-6424-4412-a531-5b9985a01e11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:23.518916 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32412a7-6424-4412-a531-5b9985a01e11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:23.524681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-f32412a7-6424-4412-a531-5b9985a01e11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:23.524681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:23.524681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:23.524681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:23.524681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:23.524681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:23.524681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:23.524681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:23.524681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:23.524681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:23.524681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:23.524681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:23.524681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:23.524681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:23.524681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:23.524681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:23.525609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:23.525609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:23.525609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:23.525609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:23.525609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:23.525609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:23.525609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:23.525609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:23.525609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:23.525609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:23.525609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:23.525609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:23.525609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:23.525609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:23.525609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:23.525609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:23.526859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:23.526859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:23.526859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:23.526859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:23.526859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:23.526859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:23.526859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:23.526859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:23.526859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:23.526859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:23.526859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:23.526859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:23.526859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 189/385] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:23.531908 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89122507-bd60-4657-8f80-d490eb00ceb2 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:23.662434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89122507-bd60-4657-8f80-d490eb00ceb2 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:23.698426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:23.706492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9072eda-782b-4530-b6ea-8f2669b6e7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:23.707141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9072eda-782b-4530-b6ea-8f2669b6e7ca None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:23.707461 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9072eda-782b-4530-b6ea-8f2669b6e7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:23.708309 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9072eda-782b-4530-b6ea-8f2669b6e7ca None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:23.709082 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9072eda-782b-4530-b6ea-8f2669b6e7ca None admin] RBAC: Adding query filter params (name=member) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:23.709391 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9072eda-782b-4530-b6ea-8f2669b6e7ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:23.710767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9072eda-782b-4530-b6ea-8f2669b6e7ca None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:23.725070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 197/386] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:23.731625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32412a7-6424-4412-a531-5b9985a01e11 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:23.735587 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:23.860243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32412a7-6424-4412-a531-5b9985a01e11 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:23.906385 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251b954-5e5c-4766-a457-1282d6665ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:23.907058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251b954-5e5c-4766-a457-1282d6665ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:23.907348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251b954-5e5c-4766-a457-1282d6665ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:23.907627 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251b954-5e5c-4766-a457-1282d6665ef2 None admin] PATH_INFO: `/v3/groups/a96ee9db2ec744d288e355aa008e72a8` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:23.908777 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251b954-5e5c-4766-a457-1282d6665ef2 None admin] RBAC: Authorizing `identity:get_group(group_id=a96ee9db2ec744d288e355aa008e72a8)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:23.910243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251b954-5e5c-4766-a457-1282d6665ef2 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:23.912409 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 190/387] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 31 19:05:23 2022] GET /identity/v3/groups/a96ee9db2ec744d288e355aa008e72a8 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:23.917996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9072eda-782b-4530-b6ea-8f2669b6e7ca None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:23.920008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251b954-5e5c-4766-a457-1282d6665ef2 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:24.041826 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:24.101403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9072eda-782b-4530-b6ea-8f2669b6e7ca None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:24.104973 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251b954-5e5c-4766-a457-1282d6665ef2 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:24.145480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc2e60-fb00-45b2-afb8-54b41cd34152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:24.146298 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc2e60-fb00-45b2-afb8-54b41cd34152 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.146809 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc2e60-fb00-45b2-afb8-54b41cd34152 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.147831 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc2e60-fb00-45b2-afb8-54b41cd34152 None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.149582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc2e60-fb00-45b2-afb8-54b41cd34152 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:24.151816 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc2e60-fb00-45b2-afb8-54b41cd34152 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:24.154316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 198/388] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:23 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:24.161641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc2e60-fb00-45b2-afb8-54b41cd34152 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:24.184533 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3f3fb-615d-4cb8-aa10-a625e0941b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:24.185407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3f3fb-615d-4cb8-aa10-a625e0941b76 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.185830 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3f3fb-615d-4cb8-aa10-a625e0941b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.186243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3f3fb-615d-4cb8-aa10-a625e0941b76 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.209734 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3f3fb-615d-4cb8-aa10-a625e0941b76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:24.215599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-fef3f3fb-615d-4cb8-aa10-a625e0941b76 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:24.215599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:24.215599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:24.215599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:24.215599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:24.215599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:24.215599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:24.215599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:24.215599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:24.215599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:24.215599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:24.215599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:24.215599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:24.215599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:24.215599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:24.215599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:24.216716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:24.216716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:24.216716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:24.216716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:24.216716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:24.216716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:24.216716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:24.216716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:24.216716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:24.216716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:24.216716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:24.216716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:24.216716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:24.216716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:24.216716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:24.216716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:24.217764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:24.217764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:24.217764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:24.217764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:24.217764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:24.217764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:24.217764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:24.217764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:24.217764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:24.217764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:24.217764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:24.217764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:24.220005 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 191/389] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:24.232355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3f3fb-615d-4cb8-aa10-a625e0941b76 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:24.243241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:24.310140 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc2e60-fb00-45b2-afb8-54b41cd34152 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:24.355380 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d18966-5478-42c1-96bf-5c48511fe449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:24.356052 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d18966-5478-42c1-96bf-5c48511fe449 None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.356327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d18966-5478-42c1-96bf-5c48511fe449 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.356598 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d18966-5478-42c1-96bf-5c48511fe449 None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69/groups/a96ee9db2ec744d288e355aa008e72a8/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d18966-5478-42c1-96bf-5c48511fe449 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69, group_id=a96ee9db2ec744d288e355aa008e72a8, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:24.365638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d18966-5478-42c1-96bf-5c48511fe449 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:24.378506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3f3fb-615d-4cb8-aa10-a625e0941b76 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:24.417157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:24.424537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b51f42-c8a7-4fdd-bc87-eb2472d6063b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:24.427885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b51f42-c8a7-4fdd-bc87-eb2472d6063b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.429633 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b51f42-c8a7-4fdd-bc87-eb2472d6063b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.430100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b51f42-c8a7-4fdd-bc87-eb2472d6063b None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.430929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b51f42-c8a7-4fdd-bc87-eb2472d6063b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:24.431278 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b51f42-c8a7-4fdd-bc87-eb2472d6063b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:24.432674 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b51f42-c8a7-4fdd-bc87-eb2472d6063b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:24.433865 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 199/390] 10.222.0.59 () {58 vars in 1442 bytes} [Wed Aug 31 19:05:24 2022] PUT /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69/groups/a96ee9db2ec744d288e355aa008e72a8/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:05:24.439961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd25f82-866d-4fbe-9179-56a371ca0336 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.440228 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd25f82-866d-4fbe-9179-56a371ca0336 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.440428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd25f82-866d-4fbe-9179-56a371ca0336 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.443824 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 200/391] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:24.450836 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1288a-fbe2-4291-af2c-4b44a29f8fe7 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.451215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1288a-fbe2-4291-af2c-4b44a29f8fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.451510 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1288a-fbe2-4291-af2c-4b44a29f8fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.454487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 192/392] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:24.460692 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b51f42-c8a7-4fdd-bc87-eb2472d6063b None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:24.495696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-b0a1288a-fbe2-4291-af2c-4b44a29f8fe7 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:24.496920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b0a1288a-fbe2-4291-af2c-4b44a29f8fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:24.597084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 201/393] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:24.632009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b51f42-c8a7-4fdd-bc87-eb2472d6063b None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:24.681022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35081494-9a88-4ac6-99d1-6524bee4bfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:24.682015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35081494-9a88-4ac6-99d1-6524bee4bfa0 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.682500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35081494-9a88-4ac6-99d1-6524bee4bfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.683032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35081494-9a88-4ac6-99d1-6524bee4bfa0 None admin] PATH_INFO: `/v3/groups/a96ee9db2ec744d288e355aa008e72a8` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.684424 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35081494-9a88-4ac6-99d1-6524bee4bfa0 None admin] RBAC: Authorizing `identity:get_group(group_id=a96ee9db2ec744d288e355aa008e72a8)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:24.686123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35081494-9a88-4ac6-99d1-6524bee4bfa0 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:24.688402 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 193/394] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 31 19:05:24 2022] GET /identity/v3/groups/a96ee9db2ec744d288e355aa008e72a8 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:24.694627 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1288a-fbe2-4291-af2c-4b44a29f8fe7 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:24.752045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:24.782635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b4be5-8415-4b57-a17f-0a3e0a0bb9ba None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.783183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b4be5-8415-4b57-a17f-0a3e0a0bb9ba None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.783183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b4be5-8415-4b57-a17f-0a3e0a0bb9ba None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.784874 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 194/395] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:24.791484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b28f0e-2d02-4ffb-a13a-6f55e4e8e2cc None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.791777 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b28f0e-2d02-4ffb-a13a-6f55e4e8e2cc None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.792056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b28f0e-2d02-4ffb-a13a-6f55e4e8e2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.826819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1288a-fbe2-4291-af2c-4b44a29f8fe7 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:24.827906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d3b28f0e-2d02-4ffb-a13a-6f55e4e8e2cc None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:24.829110 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d3b28f0e-2d02-4ffb-a13a-6f55e4e8e2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:24.899328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4397a3b-6a77-4c4d-aa47-56778e31df9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:24.899983 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4397a3b-6a77-4c4d-aa47-56778e31df9b None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.900264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4397a3b-6a77-4c4d-aa47-56778e31df9b None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.900541 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4397a3b-6a77-4c4d-aa47-56778e31df9b None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.902600 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4397a3b-6a77-4c4d-aa47-56778e31df9b None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:24.904093 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4397a3b-6a77-4c4d-aa47-56778e31df9b None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:24.907835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 202/396] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:24 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:24.915618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4397a3b-6a77-4c4d-aa47-56778e31df9b None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:24.928267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:24.947636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 195/397] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:24.948582 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b28f0e-2d02-4ffb-a13a-6f55e4e8e2cc None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:25.093396 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4397a3b-6a77-4c4d-aa47-56778e31df9b None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:25.116033 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b28f0e-2d02-4ffb-a13a-6f55e4e8e2cc None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:25.116331 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:25.142839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad817002-169f-48df-813a-55c8552a5c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:25.143898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad817002-169f-48df-813a-55c8552a5c6d None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:25.144188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad817002-169f-48df-813a-55c8552a5c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:25.144464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad817002-169f-48df-813a-55c8552a5c6d None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69/groups/a96ee9db2ec744d288e355aa008e72a8/roles/e14f37f4a3a24f8da0f757aabb1f39be` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:25.147922 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad817002-169f-48df-813a-55c8552a5c6d None admin] RBAC: Authorizing `identity:create_grant(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69, group_id=a96ee9db2ec744d288e355aa008e72a8, role_id=e14f37f4a3a24f8da0f757aabb1f39be)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:25.149497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad817002-169f-48df-813a-55c8552a5c6d None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:25.174551 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d001bb5-c3a2-4407-adc9-3977b83c8277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:25.175297 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d001bb5-c3a2-4407-adc9-3977b83c8277 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:25.175578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d001bb5-c3a2-4407-adc9-3977b83c8277 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:25.175852 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d001bb5-c3a2-4407-adc9-3977b83c8277 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:25.187416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d001bb5-c3a2-4407-adc9-3977b83c8277 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:25.193144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-2d001bb5-c3a2-4407-adc9-3977b83c8277 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:25.193144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:25.193144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:25.193144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:25.193144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:25.193144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:25.193144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:25.193144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:25.193144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:25.193144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:25.193144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:25.193144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:25.193144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:25.193144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:25.193144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:25.193144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:25.194100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:25.194100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:25.194100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:25.194100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:25.194100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:25.194100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:25.194100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:25.194100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:25.194100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:25.194100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:25.194100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:25.194100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:25.194100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:25.194100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:25.194100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:25.194100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:25.195314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:25.195314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:25.195314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:25.195314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:25.195314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:25.195314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:25.195314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:25.195314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:25.195314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:25.195314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:25.195314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:25.195314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:25.195314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 196/398] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:25.201134 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d001bb5-c3a2-4407-adc9-3977b83c8277 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:25.226668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 203/399] 10.222.0.59 () {58 vars in 1442 bytes} [Wed Aug 31 19:05:24 2022] PUT /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69/groups/a96ee9db2ec744d288e355aa008e72a8/roles/e14f37f4a3a24f8da0f757aabb1f39be => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:05:25.264723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:25.295072 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad817002-169f-48df-813a-55c8552a5c6d None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:25.344889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d001bb5-c3a2-4407-adc9-3977b83c8277 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:25.419385 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d6890-52d9-4544-85e9-6f8c5ae613e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:25.420016 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d6890-52d9-4544-85e9-6f8c5ae613e6 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:25.420292 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d6890-52d9-4544-85e9-6f8c5ae613e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:25.420562 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d6890-52d9-4544-85e9-6f8c5ae613e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:25.421242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d6890-52d9-4544-85e9-6f8c5ae613e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:25.421545 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d6890-52d9-4544-85e9-6f8c5ae613e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:25.424156 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d6890-52d9-4544-85e9-6f8c5ae613e6 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:25.436872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 197/400] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:25.443903 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d6890-52d9-4544-85e9-6f8c5ae613e6 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:25.479474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad817002-169f-48df-813a-55c8552a5c6d None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:25.517961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb754f-cc60-4b4f-b48a-82f067b043f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:25.518989 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb754f-cc60-4b4f-b48a-82f067b043f7 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:25.519462 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb754f-cc60-4b4f-b48a-82f067b043f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:25.520449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb754f-cc60-4b4f-b48a-82f067b043f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:25.532632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb754f-cc60-4b4f-b48a-82f067b043f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:25.538976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-4bfb754f-cc60-4b4f-b48a-82f067b043f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:25.538976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:25.538976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:25.538976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:25.538976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:25.538976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:25.538976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:25.538976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:25.538976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:25.538976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:25.538976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:25.538976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:25.538976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:25.538976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:25.538976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:25.538976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:25.539888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:25.539888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:25.539888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:25.539888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:25.539888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:25.539888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:25.539888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:25.539888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:25.539888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:25.539888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:25.539888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:25.539888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:25.539888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:25.539888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:25.539888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:25.539888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:25.540702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:25.540702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:25.540702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:25.540702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:25.540702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:25.540702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:25.540702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:25.540702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:25.540702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:25.540702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:25.540702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:25.540702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:25.547442 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 204/401] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:25.549849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb754f-cc60-4b4f-b48a-82f067b043f7 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:25.556829 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d6890-52d9-4544-85e9-6f8c5ae613e6 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:25.571182 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:25.590541 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:25.595364 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91566544-09bf-4aa2-b818-a0681b73e4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:25.596246 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91566544-09bf-4aa2-b818-a0681b73e4be None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:25.596749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91566544-09bf-4aa2-b818-a0681b73e4be None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:25.597221 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91566544-09bf-4aa2-b818-a0681b73e4be None admin] PATH_INFO: `/v3/groups/a96ee9db2ec744d288e355aa008e72a8` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:25.599256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91566544-09bf-4aa2-b818-a0681b73e4be None admin] RBAC: Authorizing `identity:get_group(group_id=a96ee9db2ec744d288e355aa008e72a8)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:25.600966 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91566544-09bf-4aa2-b818-a0681b73e4be None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:25.606943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 198/402] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 31 19:05:25 2022] GET /identity/v3/groups/a96ee9db2ec744d288e355aa008e72a8 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:25.609023 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91566544-09bf-4aa2-b818-a0681b73e4be None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:25.703326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb754f-cc60-4b4f-b48a-82f067b043f7 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:25.738623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91566544-09bf-4aa2-b818-a0681b73e4be None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:25.749947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f9c02-7d35-4535-8a59-9c80f558eb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:25.751317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f9c02-7d35-4535-8a59-9c80f558eb19 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:25.752206 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f9c02-7d35-4535-8a59-9c80f558eb19 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:25.752862 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f9c02-7d35-4535-8a59-9c80f558eb19 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:25.753848 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f9c02-7d35-4535-8a59-9c80f558eb19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:25.755219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f9c02-7d35-4535-8a59-9c80f558eb19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:25.757626 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f9c02-7d35-4535-8a59-9c80f558eb19 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:25.768810 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 205/403] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:25.771561 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9b6c9-213f-40a7-a55a-8ae586528fa9 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:25.772320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9b6c9-213f-40a7-a55a-8ae586528fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:25.773210 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9b6c9-213f-40a7-a55a-8ae586528fa9 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:25.775521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 206/404] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:25.780049 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9b6c9-213f-40a7-a55a-8ae586528fa9 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:25.787062 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebdf44a-74bb-474b-bf00-fe8579ba5ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:25.787774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdf44a-74bb-474b-bf00-fe8579ba5ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:25.788061 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdf44a-74bb-474b-bf00-fe8579ba5ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:25.788342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdf44a-74bb-474b-bf00-fe8579ba5ad5 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:25.789549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebdf44a-74bb-474b-bf00-fe8579ba5ad5 None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:25.791096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebdf44a-74bb-474b-bf00-fe8579ba5ad5 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:25.798043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 199/405] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:25 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:25.798043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef167049-656a-4fbf-9680-00dd3519e5ec None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:25.798043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef167049-656a-4fbf-9680-00dd3519e5ec None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:25.798043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef167049-656a-4fbf-9680-00dd3519e5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:25.820686 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-ef167049-656a-4fbf-9680-00dd3519e5ec None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:25.822467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ef167049-656a-4fbf-9680-00dd3519e5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:25.881367 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:25.914898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 200/406] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:25.914898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef167049-656a-4fbf-9680-00dd3519e5ec None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:25.939782 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:25.946405 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9b6c9-213f-40a7-a55a-8ae586528fa9 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:25.997978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4132a1e-83c1-43b5-ad5f-0ecddbfb2291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:25.998535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4132a1e-83c1-43b5-ad5f-0ecddbfb2291 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:25.998812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4132a1e-83c1-43b5-ad5f-0ecddbfb2291 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:25.999111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4132a1e-83c1-43b5-ad5f-0ecddbfb2291 None admin] PATH_INFO: `/v3/groups/a96ee9db2ec744d288e355aa008e72a8` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:26.003078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4132a1e-83c1-43b5-ad5f-0ecddbfb2291 None admin] RBAC: Authorizing `identity:get_group(group_id=a96ee9db2ec744d288e355aa008e72a8)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:26.006845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4132a1e-83c1-43b5-ad5f-0ecddbfb2291 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:26.016024 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 207/407] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 31 19:05:25 2022] GET /identity/v3/groups/a96ee9db2ec744d288e355aa008e72a8 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:26.032352 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4132a1e-83c1-43b5-ad5f-0ecddbfb2291 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:26.095190 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef167049-656a-4fbf-9680-00dd3519e5ec None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:26.132498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346cee3-52a6-4760-83eb-59cf8dc19134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:26.133167 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346cee3-52a6-4760-83eb-59cf8dc19134 None admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:26.133486 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346cee3-52a6-4760-83eb-59cf8dc19134 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:26.133761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346cee3-52a6-4760-83eb-59cf8dc19134 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d/groups/a96ee9db2ec744d288e355aa008e72a8/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:26.135719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346cee3-52a6-4760-83eb-59cf8dc19134 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca336d25ed0f4c00b1759ff3cac5302d, group_id=a96ee9db2ec744d288e355aa008e72a8, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:26.137220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346cee3-52a6-4760-83eb-59cf8dc19134 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:26.200737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4132a1e-83c1-43b5-ad5f-0ecddbfb2291 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:26.236193 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 201/408] 10.222.0.59 () {58 vars in 1442 bytes} [Wed Aug 31 19:05:25 2022] PUT /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d/groups/a96ee9db2ec744d288e355aa008e72a8/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:05:26.240051 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346cee3-52a6-4760-83eb-59cf8dc19134 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:26.246909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a80fd-93a2-4782-87d0-52e3f30431c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:26.246909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a80fd-93a2-4782-87d0-52e3f30431c6 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:26.246909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a80fd-93a2-4782-87d0-52e3f30431c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:26.246909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a80fd-93a2-4782-87d0-52e3f30431c6 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:26.247316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a80fd-93a2-4782-87d0-52e3f30431c6 None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:26.247316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a80fd-93a2-4782-87d0-52e3f30431c6 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:26.247316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 208/409] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:26 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:26.260313 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a80fd-93a2-4782-87d0-52e3f30431c6 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:26.334185 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:26.355029 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:26.450522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a80fd-93a2-4782-87d0-52e3f30431c6 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:26.452479 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346cee3-52a6-4760-83eb-59cf8dc19134 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:26.492871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730eb6b-295b-44a7-81b9-c55dae539d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:26.493522 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730eb6b-295b-44a7-81b9-c55dae539d3c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:26.493795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730eb6b-295b-44a7-81b9-c55dae539d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:26.494115 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730eb6b-295b-44a7-81b9-c55dae539d3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:26.494821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba42b3-2f3e-4eed-a957-df5765a8c5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:26.495726 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba42b3-2f3e-4eed-a957-df5765a8c5c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:26.496254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba42b3-2f3e-4eed-a957-df5765a8c5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:26.496821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba42b3-2f3e-4eed-a957-df5765a8c5c3 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d/groups/a96ee9db2ec744d288e355aa008e72a8/roles/e14f37f4a3a24f8da0f757aabb1f39be` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:26.499292 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba42b3-2f3e-4eed-a957-df5765a8c5c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca336d25ed0f4c00b1759ff3cac5302d, group_id=a96ee9db2ec744d288e355aa008e72a8, role_id=e14f37f4a3a24f8da0f757aabb1f39be)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:26.500948 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba42b3-2f3e-4eed-a957-df5765a8c5c3 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:26.504977 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730eb6b-295b-44a7-81b9-c55dae539d3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:26.510054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-0730eb6b-295b-44a7-81b9-c55dae539d3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:26.510054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:26.510054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:26.510054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:26.510054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:26.510054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:26.510054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:26.510054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:26.510054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:26.510054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:26.510054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:26.510054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:26.510054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:26.510054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:26.510054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:26.510054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:26.511068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:26.511068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:26.511068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:26.511068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:26.511068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:26.511068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:26.511068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:26.511068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:26.511068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:26.511068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:26.511068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:26.511068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:26.511068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:26.511068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:26.511068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:26.511068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:26.511904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:26.511904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:26.511904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:26.511904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:26.511904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:26.511904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:26.511904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:26.511904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:26.511904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:26.511904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:26.511904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:26.511904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:26.511904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 202/410] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:05:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:26.518858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730eb6b-295b-44a7-81b9-c55dae539d3c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:26.573936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 209/411] 10.222.0.59 () {58 vars in 1442 bytes} [Wed Aug 31 19:05:26 2022] PUT /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d/groups/a96ee9db2ec744d288e355aa008e72a8/roles/e14f37f4a3a24f8da0f757aabb1f39be => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:05:26.625751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:26.635270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730eb6b-295b-44a7-81b9-c55dae539d3c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:26.677091 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f136b46-7d62-41c8-8240-750273e759bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:26.677712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f136b46-7d62-41c8-8240-750273e759bf None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:26.677989 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f136b46-7d62-41c8-8240-750273e759bf None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:26.678322 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f136b46-7d62-41c8-8240-750273e759bf None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:26.679047 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f136b46-7d62-41c8-8240-750273e759bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:26.679352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f136b46-7d62-41c8-8240-750273e759bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:26.680784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f136b46-7d62-41c8-8240-750273e759bf None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:26.690789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 203/412] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:26.696528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba42b3-2f3e-4eed-a957-df5765a8c5c3 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:26.700409 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75603e5b-002a-4eac-8ab5-8412edfda4fa None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:26.700694 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75603e5b-002a-4eac-8ab5-8412edfda4fa None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:26.700971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75603e5b-002a-4eac-8ab5-8412edfda4fa None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:26.702317 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 204/413] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:26.708018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5f122-f71d-4001-a894-82f63a438b16 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:26.708313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5f122-f71d-4001-a894-82f63a438b16 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:26.708565 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5f122-f71d-4001-a894-82f63a438b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:26.728642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-c5a5f122-f71d-4001-a894-82f63a438b16 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:26.729861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c5a5f122-f71d-4001-a894-82f63a438b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:26.805979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 205/414] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:05:26.835623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:26.842756 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba42b3-2f3e-4eed-a957-df5765a8c5c3 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:26.888424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ae8f0-edf6-44c3-a200-44fb06c05b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:26.889071 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ae8f0-edf6-44c3-a200-44fb06c05b14 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:26.889345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ae8f0-edf6-44c3-a200-44fb06c05b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:26.889614 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ae8f0-edf6-44c3-a200-44fb06c05b14 None admin] PATH_INFO: `/v3/groups/e2a2d4a89fe640229d7c3f331570ad1f` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:26.890802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ae8f0-edf6-44c3-a200-44fb06c05b14 None admin] RBAC: Authorizing `identity:get_group(group_id=e2a2d4a89fe640229d7c3f331570ad1f)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:26.892434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ae8f0-edf6-44c3-a200-44fb06c05b14 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:26.894738 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 210/415] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 31 19:05:26 2022] GET /identity/v3/groups/e2a2d4a89fe640229d7c3f331570ad1f => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:26.901203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5f122-f71d-4001-a894-82f63a438b16 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:27.001644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:27.011304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5f122-f71d-4001-a894-82f63a438b16 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:27.047169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddcd93-6b3a-421a-a7ee-50b18803885f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:27.047870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddcd93-6b3a-421a-a7ee-50b18803885f None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:27.048149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddcd93-6b3a-421a-a7ee-50b18803885f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:27.048416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddcd93-6b3a-421a-a7ee-50b18803885f None admin] PATH_INFO: `/v3/projects/c9e5945a1a4d41fca11373bc396c02fd` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:27.049586 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddcd93-6b3a-421a-a7ee-50b18803885f None admin] RBAC: Authorizing `identity:get_project(project_id=c9e5945a1a4d41fca11373bc396c02fd)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:27.051554 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddcd93-6b3a-421a-a7ee-50b18803885f None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:27.053484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 206/416] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:26 2022] GET /identity/v3/projects/c9e5945a1a4d41fca11373bc396c02fd => generated 344 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:27.059635 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ae8f0-edf6-44c3-a200-44fb06c05b14 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:27.108469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddcd93-6b3a-421a-a7ee-50b18803885f None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:27.243489 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ae8f0-edf6-44c3-a200-44fb06c05b14 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:27.266696 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddcd93-6b3a-421a-a7ee-50b18803885f None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:27.282307 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80651792-7a55-4471-b2a1-1599834a8b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:27.286067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80651792-7a55-4471-b2a1-1599834a8b64 None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:27.286067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80651792-7a55-4471-b2a1-1599834a8b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:27.286067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80651792-7a55-4471-b2a1-1599834a8b64 None admin] PATH_INFO: `/v3/projects/c9e5945a1a4d41fca11373bc396c02fd/groups/e2a2d4a89fe640229d7c3f331570ad1f/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:27.286067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80651792-7a55-4471-b2a1-1599834a8b64 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9e5945a1a4d41fca11373bc396c02fd, group_id=e2a2d4a89fe640229d7c3f331570ad1f, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:27.287850 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80651792-7a55-4471-b2a1-1599834a8b64 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:27.323596 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc5ebf-2851-488f-b3af-a08d252a72cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:27.324289 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc5ebf-2851-488f-b3af-a08d252a72cf None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:27.324563 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc5ebf-2851-488f-b3af-a08d252a72cf None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:27.324827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc5ebf-2851-488f-b3af-a08d252a72cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:27.336753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc5ebf-2851-488f-b3af-a08d252a72cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:27.341892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-4acc5ebf-2851-488f-b3af-a08d252a72cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:27.341892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:27.341892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:27.341892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:27.341892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:27.341892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:27.341892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:27.341892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:27.341892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:27.341892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:27.341892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:27.341892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:27.341892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:27.341892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:27.341892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:27.341892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:27.355763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:27.355763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:27.355763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:27.355763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:27.355763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:27.355763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:27.355763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:27.355763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:27.355763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:27.355763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:27.355763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:27.355763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:27.355763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:27.355763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:27.355763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:27.355763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:27.356593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:27.356593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:27.356593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:27.356593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:27.356593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:27.356593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:27.356593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:27.356593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:27.356593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:27.356593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:27.356593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:27.356593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:27.356593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 207/417] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:27.356593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc5ebf-2851-488f-b3af-a08d252a72cf None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:27.374839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 211/418] 10.222.0.59 () {58 vars in 1442 bytes} [Wed Aug 31 19:05:27 2022] PUT /identity/v3/projects/c9e5945a1a4d41fca11373bc396c02fd/groups/e2a2d4a89fe640229d7c3f331570ad1f/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:05:27.381913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:27.493277 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc5ebf-2851-488f-b3af-a08d252a72cf None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:27.541515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e5879-cf9b-460c-9629-a79e8791c174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:27.542130 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e5879-cf9b-460c-9629-a79e8791c174 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:27.542406 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e5879-cf9b-460c-9629-a79e8791c174 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:27.542675 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e5879-cf9b-460c-9629-a79e8791c174 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:27.543378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e5879-cf9b-460c-9629-a79e8791c174 None admin] RBAC: Adding query filter params (name=member) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:27.543679 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e5879-cf9b-460c-9629-a79e8791c174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:27.545054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e5879-cf9b-460c-9629-a79e8791c174 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:27.555556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 208/419] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:27.561043 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80651792-7a55-4471-b2a1-1599834a8b64 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:27.591440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:27.674108 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80651792-7a55-4471-b2a1-1599834a8b64 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:27.716804 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a33502-da87-4141-b1b6-f4fe75a67de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:27.717515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a33502-da87-4141-b1b6-f4fe75a67de6 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:27.717794 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a33502-da87-4141-b1b6-f4fe75a67de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:27.718068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a33502-da87-4141-b1b6-f4fe75a67de6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:27.792833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a33502-da87-4141-b1b6-f4fe75a67de6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:27.794356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a33502-da87-4141-b1b6-f4fe75a67de6 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:27.805645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c4abd-43cf-4e21-a2b1-5bfb5ed89bfe None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:27.805963 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c4abd-43cf-4e21-a2b1-5bfb5ed89bfe None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:27.806204 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c4abd-43cf-4e21-a2b1-5bfb5ed89bfe None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:27.808750 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 209/420] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:27.812558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc3459-9595-46d1-93fc-b24631ec48f1 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:27.812839 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc3459-9595-46d1-93fc-b24631ec48f1 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:27.813111 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc3459-9595-46d1-93fc-b24631ec48f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:27.827765 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:27.847285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-ebfc3459-9595-46d1-93fc-b24631ec48f1 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:27.848937 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ebfc3459-9595-46d1-93fc-b24631ec48f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:27.905701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 212/421] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:27.914437 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a33502-da87-4141-b1b6-f4fe75a67de6 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:28.007183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 210/422] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:05:28.041981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a33502-da87-4141-b1b6-f4fe75a67de6 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:28.066103 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:28.078766 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046eeb8-9655-4a1b-ad1c-62d34743f381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:28.079632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046eeb8-9655-4a1b-ad1c-62d34743f381 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.079921 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046eeb8-9655-4a1b-ad1c-62d34743f381 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.080194 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046eeb8-9655-4a1b-ad1c-62d34743f381 None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.081354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046eeb8-9655-4a1b-ad1c-62d34743f381 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:28.083467 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046eeb8-9655-4a1b-ad1c-62d34743f381 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:28.089968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 213/423] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:27 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:28.091718 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc3459-9595-46d1-93fc-b24631ec48f1 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:28.106582 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:28.206003 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc3459-9595-46d1-93fc-b24631ec48f1 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:28.246065 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155e00b-5146-4445-8864-2bf6547a1613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:28.247047 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155e00b-5146-4445-8864-2bf6547a1613 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.247482 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155e00b-5146-4445-8864-2bf6547a1613 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.247901 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155e00b-5146-4445-8864-2bf6547a1613 None admin] PATH_INFO: `/v3/groups/a96ee9db2ec744d288e355aa008e72a8` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.249172 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155e00b-5146-4445-8864-2bf6547a1613 None admin] RBAC: Authorizing `identity:get_group(group_id=a96ee9db2ec744d288e355aa008e72a8)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:28.250748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155e00b-5146-4445-8864-2bf6547a1613 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:28.258087 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 211/424] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 31 19:05:28 2022] GET /identity/v3/groups/a96ee9db2ec744d288e355aa008e72a8 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:28.259574 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046eeb8-9655-4a1b-ad1c-62d34743f381 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:28.324672 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155e00b-5146-4445-8864-2bf6547a1613 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:28.424554 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046eeb8-9655-4a1b-ad1c-62d34743f381 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:28.472607 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:28.496648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7c1cb-23b4-4a73-9f2f-957a7c72d995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:28.497261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7c1cb-23b4-4a73-9f2f-957a7c72d995 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.497536 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7c1cb-23b4-4a73-9f2f-957a7c72d995 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.497805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7c1cb-23b4-4a73-9f2f-957a7c72d995 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.498530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7c1cb-23b4-4a73-9f2f-957a7c72d995 None admin] RBAC: Adding query filter params (group.id=a96ee9db2ec744d288e355aa008e72a8, role.id=357fdcf3417c45319b9d424a5ab4e9ca, scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:28.498832 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7c1cb-23b4-4a73-9f2f-957a7c72d995 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:28.502104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7c1cb-23b4-4a73-9f2f-957a7c72d995 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:28.510129 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155e00b-5146-4445-8864-2bf6547a1613 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:28.514752 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 214/425] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 31 19:05:28 2022] GET /identity/v3/role_assignments?group.id=a96ee9db2ec744d288e355aa008e72a8&scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69&role.id=357fdcf3417c45319b9d424a5ab4e9ca => generated 611 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:28.565139 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3fe4a-0351-448c-8437-36c0ecdfba7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:28.565967 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3fe4a-0351-448c-8437-36c0ecdfba7f None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.566244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3fe4a-0351-448c-8437-36c0ecdfba7f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.566515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3fe4a-0351-448c-8437-36c0ecdfba7f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.578893 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3fe4a-0351-448c-8437-36c0ecdfba7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:28.584980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-2de3fe4a-0351-448c-8437-36c0ecdfba7f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:28.584980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:28.584980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:28.584980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:28.584980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:28.584980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:28.584980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:28.584980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:28.584980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:28.584980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:28.584980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:28.584980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:28.584980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:28.584980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:28.584980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:28.584980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:28.585924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:28.585924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:28.585924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:28.585924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:28.585924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:28.585924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:28.585924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:28.585924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:28.585924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:28.585924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:28.585924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:28.585924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:28.585924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:28.585924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:28.585924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:28.585924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:28.586821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:28.586821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:28.586821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:28.586821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:28.586821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:28.586821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:28.586821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:28.586821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:28.586821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:28.586821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:28.586821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:28.586821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:28.586821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 212/426] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:28.592219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7c1cb-23b4-4a73-9f2f-957a7c72d995 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:28.657791 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:28.732204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7c1cb-23b4-4a73-9f2f-957a7c72d995 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:28.774566 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ff230-1848-483d-a6a7-a391ae87a010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:28.775213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ff230-1848-483d-a6a7-a391ae87a010 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.775490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ff230-1848-483d-a6a7-a391ae87a010 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.775758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ff230-1848-483d-a6a7-a391ae87a010 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.776429 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ff230-1848-483d-a6a7-a391ae87a010 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:28.776727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ff230-1848-483d-a6a7-a391ae87a010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:28.778086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ff230-1848-483d-a6a7-a391ae87a010 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:28.788423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 215/427] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:28.796038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3fe4a-0351-448c-8437-36c0ecdfba7f None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:28.920741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3fe4a-0351-448c-8437-36c0ecdfba7f None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:28.955670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1132325-c9d4-4c0f-88c7-25da99134355 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.956327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1132325-c9d4-4c0f-88c7-25da99134355 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.956327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1132325-c9d4-4c0f-88c7-25da99134355 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.957924 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 216/428] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:28.964511 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5381d43-cc74-47d3-98a8-b22dbe5d2c6a None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.964803 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5381d43-cc74-47d3-98a8-b22dbe5d2c6a None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.965081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5381d43-cc74-47d3-98a8-b22dbe5d2c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.967994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef233b-79e1-4b83-a016-b5c54c3492cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:28.968613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef233b-79e1-4b83-a016-b5c54c3492cd None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.968893 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef233b-79e1-4b83-a016-b5c54c3492cd None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.969166 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef233b-79e1-4b83-a016-b5c54c3492cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.981304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:29.000178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-c5381d43-cc74-47d3-98a8-b22dbe5d2c6a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:29.000178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c5381d43-cc74-47d3-98a8-b22dbe5d2c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:29.057933 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef233b-79e1-4b83-a016-b5c54c3492cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:29.059817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef233b-79e1-4b83-a016-b5c54c3492cd None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:29.092526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 217/429] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:29.093822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9a5ba-62cf-461e-a1f6-8501fc27ef71 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.094115 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9a5ba-62cf-461e-a1f6-8501fc27ef71 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.094382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9a5ba-62cf-461e-a1f6-8501fc27ef71 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.095792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 218/430] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:29.102670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4fe35-0d62-4629-9ca5-cd7245ea7e9a None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.102983 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4fe35-0d62-4629-9ca5-cd7245ea7e9a None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.103256 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4fe35-0d62-4629-9ca5-cd7245ea7e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.132646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-c1e4fe35-0d62-4629-9ca5-cd7245ea7e9a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:29.133868 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c1e4fe35-0d62-4629-9ca5-cd7245ea7e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:29.184876 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:29.189294 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 213/431] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:29.199884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef233b-79e1-4b83-a016-b5c54c3492cd None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:29.217118 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 219/432] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:29.255966 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:29.334985 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef233b-79e1-4b83-a016-b5c54c3492cd None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:29.370406 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58add1e3-0af5-4e61-9fd5-d8a2adba1001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:29.371066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58add1e3-0af5-4e61-9fd5-d8a2adba1001 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.371345 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58add1e3-0af5-4e61-9fd5-d8a2adba1001 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.371618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58add1e3-0af5-4e61-9fd5-d8a2adba1001 None admin] PATH_INFO: `/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.372740 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58add1e3-0af5-4e61-9fd5-d8a2adba1001 None admin] RBAC: Authorizing `identity:get_project(project_id=cc74bb8ca9f74f68a55b34b3bf3c3f69)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:29.374168 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58add1e3-0af5-4e61-9fd5-d8a2adba1001 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:29.376197 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 214/433] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:29 2022] GET /identity/v3/projects/cc74bb8ca9f74f68a55b34b3bf3c3f69 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:29.382220 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4fe35-0d62-4629-9ca5-cd7245ea7e9a None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:29.434179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58add1e3-0af5-4e61-9fd5-d8a2adba1001 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:29.505094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4fe35-0d62-4629-9ca5-cd7245ea7e9a None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:29.550729 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4923e2-383c-4ec0-9a9f-ac5a8eec48aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:29.551924 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4923e2-383c-4ec0-9a9f-ac5a8eec48aa None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.552197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4923e2-383c-4ec0-9a9f-ac5a8eec48aa None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.552921 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4923e2-383c-4ec0-9a9f-ac5a8eec48aa None admin] PATH_INFO: `/v3/groups/a96ee9db2ec744d288e355aa008e72a8` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.554339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4923e2-383c-4ec0-9a9f-ac5a8eec48aa None admin] RBAC: Authorizing `identity:get_group(group_id=a96ee9db2ec744d288e355aa008e72a8)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:29.556281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4923e2-383c-4ec0-9a9f-ac5a8eec48aa None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:29.558530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 220/434] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 31 19:05:29 2022] GET /identity/v3/groups/a96ee9db2ec744d288e355aa008e72a8 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:29.559674 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4923e2-383c-4ec0-9a9f-ac5a8eec48aa None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:29.565042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58add1e3-0af5-4e61-9fd5-d8a2adba1001 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:29.573161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:29.601992 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437cae5-2826-4ce3-9376-f742efabd222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:29.602638 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437cae5-2826-4ce3-9376-f742efabd222 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.602940 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437cae5-2826-4ce3-9376-f742efabd222 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.603212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437cae5-2826-4ce3-9376-f742efabd222 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.611872 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:29.616299 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437cae5-2826-4ce3-9376-f742efabd222 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:29.621243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-0437cae5-2826-4ce3-9376-f742efabd222 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:29.621243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:29.621243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:29.621243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:29.621243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:29.621243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:29.621243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:29.621243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:29.621243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:29.621243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:29.621243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:29.621243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:29.621243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:29.621243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:29.621243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:29.621243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:29.624053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:29.624053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:29.624053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:29.624053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:29.624053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:29.624053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:29.624053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:29.624053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:29.624053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:29.624053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:29.624053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:29.624053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:29.624053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:29.624053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:29.624053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:29.624053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:29.624953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:29.624953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:29.624953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:29.624953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:29.624953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:29.624953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:29.624953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:29.624953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:29.624953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:29.624953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:29.624953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:05:29.624953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:29.624953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 215/435] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:29.624953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437cae5-2826-4ce3-9376-f742efabd222 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:29.677645 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4923e2-383c-4ec0-9a9f-ac5a8eec48aa None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:29.713608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a6e1a-0d7e-49f3-984c-75c09332ae44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:29.714245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a6e1a-0d7e-49f3-984c-75c09332ae44 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.714526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a6e1a-0d7e-49f3-984c-75c09332ae44 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.714800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a6e1a-0d7e-49f3-984c-75c09332ae44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.726299 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a6e1a-0d7e-49f3-984c-75c09332ae44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:29.731161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-6d5a6e1a-0d7e-49f3-984c-75c09332ae44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:29.731161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:29.731161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:29.731161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:29.731161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:29.731161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:29.731161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:29.731161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:29.731161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:29.731161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:29.731161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:29.731161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:29.731161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:29.731161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:29.731161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:29.731161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:29.732079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:29.732079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:29.732079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:29.732079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:29.732079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:29.732079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:29.732079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:29.732079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:29.732079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:29.732079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:29.732079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:29.732079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:29.732079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:29.732079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:29.732079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:29.732079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:29.732894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:29.732894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:29.732894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:29.732894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:29.732894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:29.732894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:29.732894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:29.732894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:29.732894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:29.732894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:29.732894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:29.732894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:29.732894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 221/436] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:29.733602 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a6e1a-0d7e-49f3-984c-75c09332ae44 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:29.741794 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437cae5-2826-4ce3-9376-f742efabd222 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:29.778199 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07226b9-968e-429d-a792-3e4aad0ad0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:29.778802 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07226b9-968e-429d-a792-3e4aad0ad0bf None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.779337 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07226b9-968e-429d-a792-3e4aad0ad0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.779445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07226b9-968e-429d-a792-3e4aad0ad0bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.780128 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07226b9-968e-429d-a792-3e4aad0ad0bf None admin] RBAC: Adding query filter params (group.id=a96ee9db2ec744d288e355aa008e72a8, role.id=e14f37f4a3a24f8da0f757aabb1f39be, scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:29.780675 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07226b9-968e-429d-a792-3e4aad0ad0bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:29.781933 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07226b9-968e-429d-a792-3e4aad0ad0bf None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:29.789066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 216/437] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 31 19:05:29 2022] GET /identity/v3/role_assignments?group.id=a96ee9db2ec744d288e355aa008e72a8&scope.project.id=cc74bb8ca9f74f68a55b34b3bf3c3f69&role.id=e14f37f4a3a24f8da0f757aabb1f39be => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:29.789953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07226b9-968e-429d-a792-3e4aad0ad0bf None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:29.825725 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:29.856686 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a6e1a-0d7e-49f3-984c-75c09332ae44 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:29.870732 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:29.896207 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0ab8e-c4fb-4567-bf43-f247ad32d854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:29.896831 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0ab8e-c4fb-4567-bf43-f247ad32d854 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.897109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0ab8e-c4fb-4567-bf43-f247ad32d854 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.897380 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0ab8e-c4fb-4567-bf43-f247ad32d854 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.898059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0ab8e-c4fb-4567-bf43-f247ad32d854 None admin] RBAC: Adding query filter params (name=member) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:29.898351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0ab8e-c4fb-4567-bf43-f247ad32d854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:29.899784 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0ab8e-c4fb-4567-bf43-f247ad32d854 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:29.908470 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07226b9-968e-429d-a792-3e4aad0ad0bf None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:29.909890 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 222/438] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:29.911968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a3c78-8955-4b56-a4d0-7a28740fe59e None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.912255 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a3c78-8955-4b56-a4d0-7a28740fe59e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.912525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a3c78-8955-4b56-a4d0-7a28740fe59e None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.913919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 223/439] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:29.916330 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a3c78-8955-4b56-a4d0-7a28740fe59e None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:29.954307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0804521-a821-4e94-a56b-ee3176acb4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:29.956366 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0804521-a821-4e94-a56b-ee3176acb4fd None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.956366 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0804521-a821-4e94-a56b-ee3176acb4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.956366 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0804521-a821-4e94-a56b-ee3176acb4fd None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.956366 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0804521-a821-4e94-a56b-ee3176acb4fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:29.958187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0804521-a821-4e94-a56b-ee3176acb4fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:29.958511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0804521-a821-4e94-a56b-ee3176acb4fd None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:29.969225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 217/440] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:29.970192 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed7bd6-4a21-4c63-a113-f9dc19267949 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.970454 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed7bd6-4a21-4c63-a113-f9dc19267949 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.970792 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed7bd6-4a21-4c63-a113-f9dc19267949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.992148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-c0ed7bd6-4a21-4c63-a113-f9dc19267949 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:29.993335 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c0ed7bd6-4a21-4c63-a113-f9dc19267949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:30.049364 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a3c78-8955-4b56-a4d0-7a28740fe59e None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:30.084248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 218/441] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:30.087104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed7bd6-4a21-4c63-a113-f9dc19267949 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:30.100185 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a3960-578e-4152-900f-fcf234bdcc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:30.101326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a3960-578e-4152-900f-fcf234bdcc76 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.101628 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a3960-578e-4152-900f-fcf234bdcc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.101879 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a3960-578e-4152-900f-fcf234bdcc76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.113080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:30.148446 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:30.194962 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a3960-578e-4152-900f-fcf234bdcc76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:30.196497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a3960-578e-4152-900f-fcf234bdcc76 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:30.230578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed7bd6-4a21-4c63-a113-f9dc19267949 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:30.275584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13282d70-b759-4534-a5ef-11842a56ec3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:30.276344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13282d70-b759-4534-a5ef-11842a56ec3d None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.277039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13282d70-b759-4534-a5ef-11842a56ec3d None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.277039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13282d70-b759-4534-a5ef-11842a56ec3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.315254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 224/442] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 19:05:30.321700 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a3960-578e-4152-900f-fcf234bdcc76 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:30.352732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13282d70-b759-4534-a5ef-11842a56ec3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:30.353298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13282d70-b759-4534-a5ef-11842a56ec3d None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:30.377351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:30.391656 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:30.427383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a3960-578e-4152-900f-fcf234bdcc76 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:30.454643 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 219/443] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:30.455518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13282d70-b759-4534-a5ef-11842a56ec3d None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:30.463175 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a452f2-f35f-41eb-8720-9315b0849424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:30.463812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a452f2-f35f-41eb-8720-9315b0849424 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.464089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a452f2-f35f-41eb-8720-9315b0849424 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.464358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a452f2-f35f-41eb-8720-9315b0849424 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.465462 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a452f2-f35f-41eb-8720-9315b0849424 None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:30.466908 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a452f2-f35f-41eb-8720-9315b0849424 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:30.468504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 225/444] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:30 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:30.469243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a452f2-f35f-41eb-8720-9315b0849424 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:30.564432 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13282d70-b759-4534-a5ef-11842a56ec3d None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:30.568394 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a452f2-f35f-41eb-8720-9315b0849424 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:30.588082 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:30.600382 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efeb442-555a-4629-9163-615f615f3384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:30.604326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efeb442-555a-4629-9163-615f615f3384 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.604326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efeb442-555a-4629-9163-615f615f3384 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.604326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efeb442-555a-4629-9163-615f615f3384 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.604591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00aff3f-dc96-4b1e-890c-1dc4aa6cbfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:30.604591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00aff3f-dc96-4b1e-890c-1dc4aa6cbfb8 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.604591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00aff3f-dc96-4b1e-890c-1dc4aa6cbfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.604591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00aff3f-dc96-4b1e-890c-1dc4aa6cbfb8 None admin] PATH_INFO: `/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.605715 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00aff3f-dc96-4b1e-890c-1dc4aa6cbfb8 None admin] RBAC: Authorizing `identity:get_project(project_id=ca336d25ed0f4c00b1759ff3cac5302d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:30.607181 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00aff3f-dc96-4b1e-890c-1dc4aa6cbfb8 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:30.608835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 226/445] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:30 2022] GET /identity/v3/projects/ca336d25ed0f4c00b1759ff3cac5302d => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:30.609593 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00aff3f-dc96-4b1e-890c-1dc4aa6cbfb8 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:30.611559 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efeb442-555a-4629-9163-615f615f3384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:30.612232 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:30.617148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-9efeb442-555a-4629-9163-615f615f3384 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:30.617148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:30.617148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:30.617148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:30.617148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:30.617148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:30.617148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:30.617148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:30.617148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:30.617148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:30.617148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:30.617148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:30.617148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:30.617148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:30.617148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:30.617148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:30.622405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:30.622405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:30.622405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:30.622405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:30.622405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:30.622405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:30.622405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:30.622405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:30.622405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:30.622405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:30.622405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:30.622405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:30.622405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:30.622405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:30.622405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:30.622405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:30.623261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:30.623261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:30.623261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:30.623261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:30.623261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:30.623261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:30.623261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:30.623261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:30.623261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:30.623261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:30.623261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:30.623261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:30.623261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 220/446] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:05:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:30.623261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efeb442-555a-4629-9163-615f615f3384 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:30.716999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00aff3f-dc96-4b1e-890c-1dc4aa6cbfb8 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:30.719505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efeb442-555a-4629-9163-615f615f3384 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:30.752314 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd2dde-16c4-4ef5-8f98-7e3e2c1c5377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:30.752956 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd2dde-16c4-4ef5-8f98-7e3e2c1c5377 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.753241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd2dde-16c4-4ef5-8f98-7e3e2c1c5377 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.753519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd2dde-16c4-4ef5-8f98-7e3e2c1c5377 None admin] PATH_INFO: `/v3/groups/a96ee9db2ec744d288e355aa008e72a8` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.754340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbcee7-3ac7-4659-9ce8-7077598c617a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:30.754713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd2dde-16c4-4ef5-8f98-7e3e2c1c5377 None admin] RBAC: Authorizing `identity:get_group(group_id=a96ee9db2ec744d288e355aa008e72a8)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:30.754992 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbcee7-3ac7-4659-9ce8-7077598c617a None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.755276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbcee7-3ac7-4659-9ce8-7077598c617a None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.755551 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbcee7-3ac7-4659-9ce8-7077598c617a None admin] PATH_INFO: `/v3/groups/a96ee9db2ec744d288e355aa008e72a8` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.756196 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd2dde-16c4-4ef5-8f98-7e3e2c1c5377 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:30.756693 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbcee7-3ac7-4659-9ce8-7077598c617a None admin] RBAC: Authorizing `identity:get_group(group_id=a96ee9db2ec744d288e355aa008e72a8)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:30.757785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 227/447] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 31 19:05:30 2022] GET /identity/v3/groups/a96ee9db2ec744d288e355aa008e72a8 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:30.758118 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbcee7-3ac7-4659-9ce8-7077598c617a None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:30.758542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd2dde-16c4-4ef5-8f98-7e3e2c1c5377 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:30.764228 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 221/448] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 31 19:05:30 2022] GET /identity/v3/groups/a96ee9db2ec744d288e355aa008e72a8 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:30.764228 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbcee7-3ac7-4659-9ce8-7077598c617a None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:30.814246 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:30.830532 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:30.862894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd2dde-16c4-4ef5-8f98-7e3e2c1c5377 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:30.867796 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbcee7-3ac7-4659-9ce8-7077598c617a None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:30.897813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf4a0d-6354-4067-afc9-e3cad4052e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:30.898428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf4a0d-6354-4067-afc9-e3cad4052e58 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.898707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf4a0d-6354-4067-afc9-e3cad4052e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.899009 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf4a0d-6354-4067-afc9-e3cad4052e58 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.899672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf4a0d-6354-4067-afc9-e3cad4052e58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:30.899975 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf4a0d-6354-4067-afc9-e3cad4052e58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:30.901343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf4a0d-6354-4067-afc9-e3cad4052e58 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:30.902199 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc99fb-42d4-4e80-b3f9-55be43622e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:30.902807 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc99fb-42d4-4e80-b3f9-55be43622e7e None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.903111 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc99fb-42d4-4e80-b3f9-55be43622e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.903390 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc99fb-42d4-4e80-b3f9-55be43622e7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.904118 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc99fb-42d4-4e80-b3f9-55be43622e7e None admin] RBAC: Adding query filter params (group.id=a96ee9db2ec744d288e355aa008e72a8, role.id=357fdcf3417c45319b9d424a5ab4e9ca, scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:30.904429 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc99fb-42d4-4e80-b3f9-55be43622e7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:30.905866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc99fb-42d4-4e80-b3f9-55be43622e7e None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:30.909474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 228/449] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:30.910325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf4a0d-6354-4067-afc9-e3cad4052e58 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:30.912542 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 222/450] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 31 19:05:30 2022] GET /identity/v3/role_assignments?group.id=a96ee9db2ec744d288e355aa008e72a8&scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d&role.id=357fdcf3417c45319b9d424a5ab4e9ca => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:30.917803 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc99fb-42d4-4e80-b3f9-55be43622e7e None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:31.028781 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc99fb-42d4-4e80-b3f9-55be43622e7e None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:31.030434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf4a0d-6354-4067-afc9-e3cad4052e58 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:31.043992 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:31.063313 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:31.065786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92c2ed-fd7e-419f-a82b-df446875f348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:31.068452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92c2ed-fd7e-419f-a82b-df446875f348 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:31.068452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92c2ed-fd7e-419f-a82b-df446875f348 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:31.068452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92c2ed-fd7e-419f-a82b-df446875f348 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:31.070364 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cfee7-63bd-4a31-b878-cb48fb4cf815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:31.071459 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cfee7-63bd-4a31-b878-cb48fb4cf815 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:31.071459 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cfee7-63bd-4a31-b878-cb48fb4cf815 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:31.072510 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cfee7-63bd-4a31-b878-cb48fb4cf815 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:31.072510 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cfee7-63bd-4a31-b878-cb48fb4cf815 None admin] RBAC: Adding query filter params (group.id=a96ee9db2ec744d288e355aa008e72a8, role.id=e14f37f4a3a24f8da0f757aabb1f39be, scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:31.072662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cfee7-63bd-4a31-b878-cb48fb4cf815 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:31.074125 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cfee7-63bd-4a31-b878-cb48fb4cf815 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:31.086456 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 229/451] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 31 19:05:30 2022] GET /identity/v3/role_assignments?group.id=a96ee9db2ec744d288e355aa008e72a8&scope.project.id=ca336d25ed0f4c00b1759ff3cac5302d&role.id=e14f37f4a3a24f8da0f757aabb1f39be => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:31.152285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92c2ed-fd7e-419f-a82b-df446875f348 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:31.153738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92c2ed-fd7e-419f-a82b-df446875f348 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:31.272659 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 223/452] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:31.277950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cfee7-63bd-4a31-b878-cb48fb4cf815 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:31.388099 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cfee7-63bd-4a31-b878-cb48fb4cf815 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:31.421712 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caefb36-a3c0-49d9-98ff-5e57bf81718f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:31.422270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caefb36-a3c0-49d9-98ff-5e57bf81718f None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:31.422548 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caefb36-a3c0-49d9-98ff-5e57bf81718f None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:31.422824 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caefb36-a3c0-49d9-98ff-5e57bf81718f None admin] PATH_INFO: `/v3/projects/c9e5945a1a4d41fca11373bc396c02fd` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:31.424008 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caefb36-a3c0-49d9-98ff-5e57bf81718f None admin] RBAC: Authorizing `identity:get_project(project_id=c9e5945a1a4d41fca11373bc396c02fd)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:31.425434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caefb36-a3c0-49d9-98ff-5e57bf81718f None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:31.427794 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 230/453] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:31 2022] GET /identity/v3/projects/c9e5945a1a4d41fca11373bc396c02fd => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:31.433374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92c2ed-fd7e-419f-a82b-df446875f348 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:31.449932 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:31.534036 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92c2ed-fd7e-419f-a82b-df446875f348 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:31.567383 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea469ea-ad17-4892-be24-fc0e519057ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:31.568002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea469ea-ad17-4892-be24-fc0e519057ed None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:31.568282 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea469ea-ad17-4892-be24-fc0e519057ed None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:31.568556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea469ea-ad17-4892-be24-fc0e519057ed None admin] PATH_INFO: `/v3/groups/e2a2d4a89fe640229d7c3f331570ad1f` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:31.569656 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea469ea-ad17-4892-be24-fc0e519057ed None admin] RBAC: Authorizing `identity:get_group(group_id=e2a2d4a89fe640229d7c3f331570ad1f)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:31.571268 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea469ea-ad17-4892-be24-fc0e519057ed None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:31.572681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 224/454] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 31 19:05:31 2022] GET /identity/v3/groups/e2a2d4a89fe640229d7c3f331570ad1f => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:31.578497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caefb36-a3c0-49d9-98ff-5e57bf81718f None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:31.639694 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:31.676616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caefb36-a3c0-49d9-98ff-5e57bf81718f None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:31.709431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a51850-1f3a-4512-a044-7a2269da973a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:31.710025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a51850-1f3a-4512-a044-7a2269da973a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:31.710304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a51850-1f3a-4512-a044-7a2269da973a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:31.710579 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a51850-1f3a-4512-a044-7a2269da973a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:31.711319 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a51850-1f3a-4512-a044-7a2269da973a None admin] RBAC: Adding query filter params (group.id=e2a2d4a89fe640229d7c3f331570ad1f, role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=c9e5945a1a4d41fca11373bc396c02fd) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:31.711627 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a51850-1f3a-4512-a044-7a2269da973a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:31.713033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a51850-1f3a-4512-a044-7a2269da973a None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:31.718608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 231/455] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 31 19:05:31 2022] GET /identity/v3/role_assignments?group.id=e2a2d4a89fe640229d7c3f331570ad1f&scope.project.id=c9e5945a1a4d41fca11373bc396c02fd&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:34.089818 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdab45c-5893-43cb-8847-4ba2bc4c72e7 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:34.090378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdab45c-5893-43cb-8847-4ba2bc4c72e7 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:34.090378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdab45c-5893-43cb-8847-4ba2bc4c72e7 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:34.091959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 225/456] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:34.097336 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928db30-b4e9-4e71-b6e4-6cbff150c51e None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:34.098378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928db30-b4e9-4e71-b6e4-6cbff150c51e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:34.098502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928db30-b4e9-4e71-b6e4-6cbff150c51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:34.127883 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-9928db30-b4e9-4e71-b6e4-6cbff150c51e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:34.129642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-9928db30-b4e9-4e71-b6e4-6cbff150c51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:34.191730 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945844d-f947-4095-a9cc-284de3ebb12a None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:34.192357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945844d-f947-4095-a9cc-284de3ebb12a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:34.192824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945844d-f947-4095-a9cc-284de3ebb12a None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:34.194619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 226/457] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:34.199707 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772af98-3b10-45ef-b640-12fcd6d3ff12 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:34.200206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772af98-3b10-45ef-b640-12fcd6d3ff12 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:34.200692 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772af98-3b10-45ef-b640-12fcd6d3ff12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:34.218155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 232/458] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:05:34.239978 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-6772af98-3b10-45ef-b640-12fcd6d3ff12 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:34.241190 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6772af98-3b10-45ef-b640-12fcd6d3ff12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:34.352226 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 227/459] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:34.636485 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928db30-b4e9-4e71-b6e4-6cbff150c51e None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:34.685536 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772af98-3b10-45ef-b640-12fcd6d3ff12 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:34.700993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:34.714247 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:34.757635 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928db30-b4e9-4e71-b6e4-6cbff150c51e None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:34.808183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ed691-f386-405c-b6fb-e59879930ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:34.809090 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ed691-f386-405c-b6fb-e59879930ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:34.809495 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ed691-f386-405c-b6fb-e59879930ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:34.809888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ed691-f386-405c-b6fb-e59879930ce4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:34.824646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ed691-f386-405c-b6fb-e59879930ce4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:34.826857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ed691-f386-405c-b6fb-e59879930ce4 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:34.849183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-577ed691-f386-405c-b6fb-e59879930ce4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:34.849183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:34.849183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:34.849183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:34.849183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:34.849183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:34.849183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:34.849183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:34.849183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:34.849183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:34.849183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:34.849183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:34.849183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:34.849183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:34.849183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:34.849183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:34.850363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:34.850363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:34.850363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:34.850363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:34.850363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:34.850363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:34.850363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:34.850363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:34.850363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:34.850363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:34.850363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:34.850363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:34.850363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:34.850363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:34.850363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:34.850363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:34.850363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:34.851486 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:34.851486 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:34.851486 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:34.857686 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772af98-3b10-45ef-b640-12fcd6d3ff12 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:34.857875 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 233/460] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:34.859329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59b3ea-f424-45cb-90d8-127efb93f1c6 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:34.859769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59b3ea-f424-45cb-90d8-127efb93f1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:34.860183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59b3ea-f424-45cb-90d8-127efb93f1c6 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:34.862088 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 234/461] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:34.863897 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59b3ea-f424-45cb-90d8-127efb93f1c6 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:34.899779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b751c-3b5a-45f4-8701-1de7de8471a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:34.900569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b751c-3b5a-45f4-8701-1de7de8471a1 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:34.900943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b751c-3b5a-45f4-8701-1de7de8471a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:34.901311 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b751c-3b5a-45f4-8701-1de7de8471a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:34.915402 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b751c-3b5a-45f4-8701-1de7de8471a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:34.916981 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b751c-3b5a-45f4-8701-1de7de8471a1 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:34.934511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-979b751c-3b5a-45f4-8701-1de7de8471a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:34.934511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:34.934511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:34.934511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:34.934511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:34.934511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:34.934511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:34.934511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:34.934511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:34.934511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:34.934511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:34.934511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:34.934511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:34.934511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:34.934511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:34.934511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:34.935518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:34.935518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:34.935518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:34.935518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:34.935518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:34.935518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:34.935518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:34.935518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:34.935518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:34.935518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:34.935518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:34.935518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:34.935518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:34.935518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:34.935518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:34.935518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:34.935518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:34.936491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:34.936491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:34.936491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:34.936491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 228/462] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:34.940665 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9b9e7-fa3d-4226-934f-981bf0038661 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:34.940665 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9b9e7-fa3d-4226-934f-981bf0038661 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:34.940665 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9b9e7-fa3d-4226-934f-981bf0038661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:34.958747 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-27e9b9e7-fa3d-4226-934f-981bf0038661 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:34.959957 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-27e9b9e7-fa3d-4226-934f-981bf0038661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:34.986723 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59b3ea-f424-45cb-90d8-127efb93f1c6 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:34.998834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:35.022653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:35.026509 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c230691-d155-4b6d-8c05-82c6c4020045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:35.027439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c230691-d155-4b6d-8c05-82c6c4020045 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.027719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c230691-d155-4b6d-8c05-82c6c4020045 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.027992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c230691-d155-4b6d-8c05-82c6c4020045 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.028633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c230691-d155-4b6d-8c05-82c6c4020045 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:35.028936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c230691-d155-4b6d-8c05-82c6c4020045 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:35.030306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c230691-d155-4b6d-8c05-82c6c4020045 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:35.032455 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 229/463] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:35.033921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6aec4-0481-4b0e-8420-cab9b182ad69 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.039998 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6aec4-0481-4b0e-8420-cab9b182ad69 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.039998 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6aec4-0481-4b0e-8420-cab9b182ad69 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.039998 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 230/464] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:35.041201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6aec4-0481-4b0e-8420-cab9b182ad69 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:35.052674 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 235/465] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:35.054215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f4c46-638b-42bf-a870-26a738fce5bf None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.054686 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f4c46-638b-42bf-a870-26a738fce5bf None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.055184 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f4c46-638b-42bf-a870-26a738fce5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.076279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-d12f4c46-638b-42bf-a870-26a738fce5bf None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:35.077471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d12f4c46-638b-42bf-a870-26a738fce5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:35.158963 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 236/466] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:35.158963 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f4c46-638b-42bf-a870-26a738fce5bf None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:35.161890 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6aec4-0481-4b0e-8420-cab9b182ad69 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:35.228012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b20186-07c2-4016-aa8b-c2230c1a81b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:35.228631 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b20186-07c2-4016-aa8b-c2230c1a81b5 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.228907 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b20186-07c2-4016-aa8b-c2230c1a81b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.229173 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b20186-07c2-4016-aa8b-c2230c1a81b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.229819 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b20186-07c2-4016-aa8b-c2230c1a81b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:35.230117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b20186-07c2-4016-aa8b-c2230c1a81b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:35.231511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b20186-07c2-4016-aa8b-c2230c1a81b5 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:35.261800 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 231/467] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:35.272364 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b20186-07c2-4016-aa8b-c2230c1a81b5 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:35.313280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:35.321204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f4c46-638b-42bf-a870-26a738fce5bf None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:35.353271 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:35.376641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79a4f5-bfb7-467c-b574-137b307eb81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:35.377257 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79a4f5-bfb7-467c-b574-137b307eb81d None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.378111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79a4f5-bfb7-467c-b574-137b307eb81d None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.379609 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79a4f5-bfb7-467c-b574-137b307eb81d None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.380549 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e79a4f5-bfb7-467c-b574-137b307eb81d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:35.382384 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e79a4f5-bfb7-467c-b574-137b307eb81d None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:35.403012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b20186-07c2-4016-aa8b-c2230c1a81b5 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:35.439882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fdb1d6-abfc-47f6-9e93-c9f2e552a0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:35.440411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdb1d6-abfc-47f6-9e93-c9f2e552a0cc None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.440691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdb1d6-abfc-47f6-9e93-c9f2e552a0cc None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.440963 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdb1d6-abfc-47f6-9e93-c9f2e552a0cc None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.442225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fdb1d6-abfc-47f6-9e93-c9f2e552a0cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:35.443696 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fdb1d6-abfc-47f6-9e93-c9f2e552a0cc None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:35.455300 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 237/468] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:05:35 2022] POST /identity/v3/users => generated 246 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:35.456642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79a4f5-bfb7-467c-b574-137b307eb81d None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:35.511895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 232/469] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:05:35 2022] POST /identity/v3/users => generated 248 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:05:35.513360 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee45ec-0bb4-4478-bfde-3ea8257f3c27 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.513710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee45ec-0bb4-4478-bfde-3ea8257f3c27 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.513991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee45ec-0bb4-4478-bfde-3ea8257f3c27 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.520567 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 233/470] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:35.521758 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee45ec-0bb4-4478-bfde-3ea8257f3c27 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:35.585066 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79a4f5-bfb7-467c-b574-137b307eb81d None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:35.624834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70063c7-09d7-4a10-bafa-561acc779b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:35.625598 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70063c7-09d7-4a10-bafa-561acc779b13 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.625887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70063c7-09d7-4a10-bafa-561acc779b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.626165 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70063c7-09d7-4a10-bafa-561acc779b13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.640055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70063c7-09d7-4a10-bafa-561acc779b13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:35.641547 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70063c7-09d7-4a10-bafa-561acc779b13 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:35.658138 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee45ec-0bb4-4478-bfde-3ea8257f3c27 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:35.658856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-d70063c7-09d7-4a10-bafa-561acc779b13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:35.658856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:35.658856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:35.658856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:35.658856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:35.658856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:35.658856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:35.658856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:35.658856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:35.658856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:35.658856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:35.658856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:35.658856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:35.658856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:35.658856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:35.658856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:35.659763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:35.659763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:35.659763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:35.659763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:35.659763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:35.659763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:35.659763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:35.659763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:35.659763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:35.659763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:35.659763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:35.659763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:35.659763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:35.659763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:35.659763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:35.659763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:35.659763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:35.661464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:35.661464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:35.661464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:35.663016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 238/471] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:35.664578 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37389d-2f85-4cb5-b8bc-c55e9b2e612b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.665088 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37389d-2f85-4cb5-b8bc-c55e9b2e612b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.665569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37389d-2f85-4cb5-b8bc-c55e9b2e612b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.670909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:35.685961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-8c37389d-2f85-4cb5-b8bc-c55e9b2e612b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:35.688635 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8c37389d-2f85-4cb5-b8bc-c55e9b2e612b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:35.689314 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:35.705183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f99d01-8b41-4af2-ab40-748b3629f7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:35.705764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f99d01-8b41-4af2-ab40-748b3629f7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.706043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f99d01-8b41-4af2-ab40-748b3629f7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.706318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f99d01-8b41-4af2-ab40-748b3629f7b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.721586 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f99d01-8b41-4af2-ab40-748b3629f7b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:35.729077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f99d01-8b41-4af2-ab40-748b3629f7b0 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:35.742647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-16f99d01-8b41-4af2-ab40-748b3629f7b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:35.742647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:35.742647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:35.742647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:35.742647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:35.742647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:35.742647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:35.742647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:35.742647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:35.742647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:35.742647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:35.742647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:35.742647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:35.742647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:35.742647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:35.742647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:35.743714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:35.743714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:35.743714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:35.743714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:35.743714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:35.743714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:35.743714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:35.743714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:35.743714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:35.743714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:35.743714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:35.743714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:35.743714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:35.743714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:35.743714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:35.743714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:35.743714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:35.744599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:35.744599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:35.744599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:35.747067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 234/472] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:35.748435 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f99d01-8b41-4af2-ab40-748b3629f7b0 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:35.777406 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 239/473] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:35.778301 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37389d-2f85-4cb5-b8bc-c55e9b2e612b None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:35.896283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f99d01-8b41-4af2-ab40-748b3629f7b0 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:35.937097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb0eee-db3c-4eca-b397-05bc5c82ac0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:35.937620 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37389d-2f85-4cb5-b8bc-c55e9b2e612b None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:35.938392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb0eee-db3c-4eca-b397-05bc5c82ac0b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.938924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb0eee-db3c-4eca-b397-05bc5c82ac0b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.939601 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb0eee-db3c-4eca-b397-05bc5c82ac0b None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.940530 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb0eee-db3c-4eca-b397-05bc5c82ac0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:35.941061 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb0eee-db3c-4eca-b397-05bc5c82ac0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:35.943354 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb0eee-db3c-4eca-b397-05bc5c82ac0b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:35.959797 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 235/474] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:35.965023 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb0eee-db3c-4eca-b397-05bc5c82ac0b None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:35.995097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:36.036229 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a47930-abe3-427d-84ff-48bdc34c877e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:36.036854 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a47930-abe3-427d-84ff-48bdc34c877e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:36.037129 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a47930-abe3-427d-84ff-48bdc34c877e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:36.037392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a47930-abe3-427d-84ff-48bdc34c877e None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:36.038034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a47930-abe3-427d-84ff-48bdc34c877e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:36.038333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a47930-abe3-427d-84ff-48bdc34c877e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:36.039734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a47930-abe3-427d-84ff-48bdc34c877e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:36.055871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:36.069421 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 240/475] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:36.074900 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a47930-abe3-427d-84ff-48bdc34c877e None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:36.089914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb0eee-db3c-4eca-b397-05bc5c82ac0b None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:36.163101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdf41b-9920-4698-bf89-d6cf8ccb63e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:36.165343 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdf41b-9920-4698-bf89-d6cf8ccb63e9 None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:36.167117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdf41b-9920-4698-bf89-d6cf8ccb63e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:36.167604 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdf41b-9920-4698-bf89-d6cf8ccb63e9 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:36.168765 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbdf41b-9920-4698-bf89-d6cf8ccb63e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:36.172517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbdf41b-9920-4698-bf89-d6cf8ccb63e9 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:36.227597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a47930-abe3-427d-84ff-48bdc34c877e None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:36.252375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 236/476] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:05:35 2022] POST /identity/v3/users => generated 248 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:36.254885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdf41b-9920-4698-bf89-d6cf8ccb63e9 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:36.267955 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cadd54-d8a7-4d97-9320-e60f2f21d82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:36.268593 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cadd54-d8a7-4d97-9320-e60f2f21d82d None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:36.268918 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cadd54-d8a7-4d97-9320-e60f2f21d82d None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:36.272574 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cadd54-d8a7-4d97-9320-e60f2f21d82d None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:36.272574 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cadd54-d8a7-4d97-9320-e60f2f21d82d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:36.272797 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cadd54-d8a7-4d97-9320-e60f2f21d82d None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:36.371317 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:36.395947 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdf41b-9920-4698-bf89-d6cf8ccb63e9 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:36.437183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0309ca2-a35a-408e-8d79-24e587620df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:36.437808 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0309ca2-a35a-408e-8d79-24e587620df9 None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:36.438094 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0309ca2-a35a-408e-8d79-24e587620df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:36.438373 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0309ca2-a35a-408e-8d79-24e587620df9 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:36.439079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0309ca2-a35a-408e-8d79-24e587620df9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:36.440465 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0309ca2-a35a-408e-8d79-24e587620df9 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:36.471488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.sql.core [None req-b0309ca2-a35a-408e-8d79-24e587620df9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 19:05:36.471488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 19:05:36.471488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [parameters: {'id': 'e2fd0b7b1df64074b3197414e5b8f61d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 19:05:36.471488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81868) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 19:05:36.479111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 241/477] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:05:36 2022] POST /identity/v3/users => generated 249 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:05:36.489549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-b0309ca2-a35a-408e-8d79-24e587620df9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 19:05:36.489549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:36.489549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 19:05:36.489549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 19:05:36.489549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 19:05:36.489549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 19:05:36.489549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 19:05:36.489549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 19:05:36.489549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 19:05:36.489549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application conn.query(q) Aug 31 19:05:36.489549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 19:05:36.489549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 19:05:36.489549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 19:05:36.489549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application result.read() Aug 31 19:05:36.489549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return ref.to_dict() Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 19:05:36.495044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application next(self.gen) Aug 31 19:05:36.495993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 19:05:36.495993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application yield resource Aug 31 19:05:36.495993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 19:05:36.495993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application next(self.gen) Aug 31 19:05:36.495993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 19:05:36.495993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 19:05:36.495993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 19:05:36.495993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 19:05:36.495993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 19:05:36.495993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise self.value Aug 31 19:05:36.495993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 19:05:36.495993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 19:05:36.495993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 19:05:36.495993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application session.commit() Aug 31 19:05:36.495993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 19:05:36.495993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 19:05:36.495993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 19:05:36.496889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 19:05:36.496889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 19:05:36.496889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self.session.flush() Aug 31 19:05:36.496889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 19:05:36.496889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 19:05:36.496889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 19:05:36.496889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 19:05:36.496889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 19:05:36.496889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application compat.raise_( Aug 31 19:05:36.496889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 19:05:36.496889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception Aug 31 19:05:36.496889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 19:05:36.496889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 19:05:36.496889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 19:05:36.496889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 19:05:36.496889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 19:05:36.496889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 19:05:36.497824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 19:05:36.497824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 19:05:36.497824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 19:05:36.497824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 19:05:36.497824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 19:05:36.497824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 19:05:36.497824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 19:05:36.497824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 19:05:36.497824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 19:05:36.497824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 19:05:36.497824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 19:05:36.497824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 19:05:36.497824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 19:05:36.497824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 19:05:36.497824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 19:05:36.497824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception Aug 31 19:05:36.498710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 19:05:36.498710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 19:05:36.498710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 19:05:36.498710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 19:05:36.498710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 19:05:36.498710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 19:05:36.498710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 19:05:36.498710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application conn.query(q) Aug 31 19:05:36.498710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 19:05:36.498710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 19:05:36.498710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 19:05:36.498710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application result.read() Aug 31 19:05:36.498710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 19:05:36.498710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 19:05:36.498710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 19:05:36.498710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 19:05:36.499592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 19:05:36.499592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 19:05:36.499592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 19:05:36.499592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 19:05:36.499592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 19:05:36.499592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 19:05:36.499592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [parameters: {'id': 'e2fd0b7b1df64074b3197414e5b8f61d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 19:05:36.499592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 19:05:36.499592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:36.499592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:36.499592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:36.499592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:36.499592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:36.499592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:36.499592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:36.499592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:36.500513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:36.500513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:36.500513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:36.500513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:36.500513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:36.500513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:36.500513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 31 19:05:36.500513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 31 19:05:36.500513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:36.500513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:36.500513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 31 19:05:36.500513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 31 19:05:36.500513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 19:05:36.500513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 19:05:36.500513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 19:05:36.500513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:36.501581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 237/478] 10.222.0.59 () {60 vars in 1131 bytes} [Wed Aug 31 19:05:36 2022] POST /identity/v3/roles => generated 144 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:36.501668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cadd54-d8a7-4d97-9320-e60f2f21d82d None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:36.575787 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:36.632740 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cadd54-d8a7-4d97-9320-e60f2f21d82d None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:36.674430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da61c8-567c-42e6-8c4e-3eab47a23cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:36.675104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da61c8-567c-42e6-8c4e-3eab47a23cfb None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:36.675381 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da61c8-567c-42e6-8c4e-3eab47a23cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:36.676209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da61c8-567c-42e6-8c4e-3eab47a23cfb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:36.691901 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da61c8-567c-42e6-8c4e-3eab47a23cfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:36.698805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-17da61c8-567c-42e6-8c4e-3eab47a23cfb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:36.698805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:36.698805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:36.698805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:36.698805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:36.698805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:36.698805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:36.698805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:36.698805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:36.698805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:36.698805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:36.698805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:36.698805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:36.698805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:36.698805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:36.698805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:36.702219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:36.702219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:36.702219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:36.702219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:36.702219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:36.702219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:36.702219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:36.702219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:36.702219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:36.702219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:36.702219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:36.702219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:36.702219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:36.702219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:36.702219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:36.702219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:36.703084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:36.703084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:36.703084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:36.703084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:36.703084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:36.703084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:36.703084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:36.703084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:36.703084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:36.703084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:36.703084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:05:36.703084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:36.703084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 242/479] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:36.705044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0309ca2-a35a-408e-8d79-24e587620df9 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:36.829036 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0309ca2-a35a-408e-8d79-24e587620df9 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:36.867581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e46ff-c3a9-4df3-94a0-c00b96b763ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:36.868278 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e46ff-c3a9-4df3-94a0-c00b96b763ff None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:36.868563 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e46ff-c3a9-4df3-94a0-c00b96b763ff None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:36.868838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e46ff-c3a9-4df3-94a0-c00b96b763ff None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:36.869538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e46ff-c3a9-4df3-94a0-c00b96b763ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:36.869850 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e46ff-c3a9-4df3-94a0-c00b96b763ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:36.871309 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e46ff-c3a9-4df3-94a0-c00b96b763ff None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:36.881091 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 238/480] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:37.747304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c3720-99ff-4f69-ae90-2797a9f7d13c None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:37.748338 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c3720-99ff-4f69-ae90-2797a9f7d13c None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:37.748831 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c3720-99ff-4f69-ae90-2797a9f7d13c None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:37.750394 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 243/481] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:37.756457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b4663-1577-4753-a418-4ac6ae3b542e None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:37.757337 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b4663-1577-4753-a418-4ac6ae3b542e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:37.757821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b4663-1577-4753-a418-4ac6ae3b542e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:37.772332 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:37.788079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-2e9b4663-1577-4753-a418-4ac6ae3b542e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:37.789718 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-2e9b4663-1577-4753-a418-4ac6ae3b542e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:37.829928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2888de-faa3-423e-88fe-4a2b4bb2a520 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:37.830201 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2888de-faa3-423e-88fe-4a2b4bb2a520 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:37.830476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2888de-faa3-423e-88fe-4a2b4bb2a520 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:37.832441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 244/482] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:37.837945 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cf9eb-444a-4ec4-97be-f57f7b6d6305 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:37.838234 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cf9eb-444a-4ec4-97be-f57f7b6d6305 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:37.838503 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cf9eb-444a-4ec4-97be-f57f7b6d6305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:37.859666 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-de8cf9eb-444a-4ec4-97be-f57f7b6d6305 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:37.860879 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-de8cf9eb-444a-4ec4-97be-f57f7b6d6305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:37.913482 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 239/483] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:37.932721 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 245/484] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:38.219962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b4663-1577-4753-a418-4ac6ae3b542e None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:38.345709 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b4663-1577-4753-a418-4ac6ae3b542e None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:38.392528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbc484-adaa-45e9-becf-0cdabe5f2f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:38.393191 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbc484-adaa-45e9-becf-0cdabe5f2f14 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:38.393475 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbc484-adaa-45e9-becf-0cdabe5f2f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:38.393752 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbc484-adaa-45e9-becf-0cdabe5f2f14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:38.396404 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cf9eb-444a-4ec4-97be-f57f7b6d6305 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:38.398321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:38.408919 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acbc484-adaa-45e9-becf-0cdabe5f2f14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:38.416872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-0acbc484-adaa-45e9-becf-0cdabe5f2f14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:38.416872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:38.416872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:38.416872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:38.416872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:38.416872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:38.416872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:38.416872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:38.416872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:38.416872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:38.416872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:38.416872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:38.416872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:38.416872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:38.416872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:38.416872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:38.417833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:38.417833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:38.417833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:38.417833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:38.417833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:38.417833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:38.417833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:38.417833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:38.417833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:38.417833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:38.417833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:38.417833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:38.417833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:38.417833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:38.417833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:38.417833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:38.418660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:38.418660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:38.418660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:38.418660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:38.418660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:38.418660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:38.418660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:38.418660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:38.418660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:38.418660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:38.418660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:38.418660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:38.418660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 240/485] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:38.431634 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbc484-adaa-45e9-becf-0cdabe5f2f14 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:38.505178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:38.609149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbc484-adaa-45e9-becf-0cdabe5f2f14 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:38.650321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37541f8b-0be7-4df9-adac-3c544fc10c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:38.650680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cf9eb-444a-4ec4-97be-f57f7b6d6305 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:38.650973 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37541f8b-0be7-4df9-adac-3c544fc10c61 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:38.651252 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37541f8b-0be7-4df9-adac-3c544fc10c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:38.651520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37541f8b-0be7-4df9-adac-3c544fc10c61 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:38.652452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37541f8b-0be7-4df9-adac-3c544fc10c61 None admin] RBAC: Adding query filter params (name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:38.652752 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37541f8b-0be7-4df9-adac-3c544fc10c61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:38.654150 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37541f8b-0be7-4df9-adac-3c544fc10c61 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:38.666836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 241/486] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:05:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:38.672217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37541f8b-0be7-4df9-adac-3c544fc10c61 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:38.715769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d3c0e-9b8c-4ceb-ac64-e99e13a5e355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:38.716456 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d3c0e-9b8c-4ceb-ac64-e99e13a5e355 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:38.716956 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d3c0e-9b8c-4ceb-ac64-e99e13a5e355 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:38.717231 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d3c0e-9b8c-4ceb-ac64-e99e13a5e355 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:38.729751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d3c0e-9b8c-4ceb-ac64-e99e13a5e355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:38.734988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-8c3d3c0e-9b8c-4ceb-ac64-e99e13a5e355 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:38.734988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:38.734988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:38.734988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:38.734988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:38.734988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:38.734988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:38.734988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:38.734988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:38.734988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:38.734988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:38.734988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:38.734988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:38.734988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:38.734988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:38.734988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:38.735910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:38.735910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:38.735910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:38.735910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:38.735910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:38.735910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:38.735910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:38.735910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:38.735910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:38.735910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:38.735910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:38.735910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:38.735910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:38.735910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:38.735910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:38.735910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:38.736737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:38.736737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:38.736737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:38.736737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:38.736737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:38.736737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:38.736737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:38.736737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:38.736737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:38.736737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:38.736737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:38.736737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:38.736737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 246/487] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:38.742573 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d3c0e-9b8c-4ceb-ac64-e99e13a5e355 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:38.793214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:38.890591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37541f8b-0be7-4df9-adac-3c544fc10c61 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:38.899729 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d3c0e-9b8c-4ceb-ac64-e99e13a5e355 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:38.902305 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:38.941444 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569374e1-077c-4fc8-b588-89a13b707810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:38.942146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569374e1-077c-4fc8-b588-89a13b707810 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:38.942424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569374e1-077c-4fc8-b588-89a13b707810 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:38.942691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569374e1-077c-4fc8-b588-89a13b707810 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:38.944460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569374e1-077c-4fc8-b588-89a13b707810 None admin] RBAC: Adding query filter params (name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:38.944781 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569374e1-077c-4fc8-b588-89a13b707810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:38.946340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569374e1-077c-4fc8-b588-89a13b707810 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:38.949474 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d11f3-fcff-4d4f-a2a2-e5bdcf53a77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:38.950196 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d11f3-fcff-4d4f-a2a2-e5bdcf53a77f None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:38.950481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d11f3-fcff-4d4f-a2a2-e5bdcf53a77f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:38.950749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d11f3-fcff-4d4f-a2a2-e5bdcf53a77f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:38.961223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 247/488] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:05:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:38.962573 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11c703-664f-44d2-a2bc-60c0c538e9aa None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:38.962887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11c703-664f-44d2-a2bc-60c0c538e9aa None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:38.963173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11c703-664f-44d2-a2bc-60c0c538e9aa None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:38.963625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d11f3-fcff-4d4f-a2a2-e5bdcf53a77f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:38.964694 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 248/489] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:38.965304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d11f3-fcff-4d4f-a2a2-e5bdcf53a77f None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:38.965412 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11c703-664f-44d2-a2bc-60c0c538e9aa None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:38.988769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-6d1d11f3-fcff-4d4f-a2a2-e5bdcf53a77f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:38.988769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:38.988769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:38.988769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:38.988769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:38.988769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:38.988769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:38.988769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:38.988769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:38.988769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:38.988769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:38.988769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:38.988769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:38.988769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:38.988769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:38.988769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:38.989762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:38.989762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:38.989762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:38.989762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:38.989762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:38.989762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:38.989762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:38.989762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:38.989762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:38.989762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:38.989762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:38.989762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:38.989762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:38.989762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:38.989762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:38.989762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:38.989762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:38.990623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:38.990623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:38.990623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:38.990623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 242/490] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:38.992423 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b20176-4116-4030-80f3-7757faf68108 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:38.993175 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b20176-4116-4030-80f3-7757faf68108 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:38.993510 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b20176-4116-4030-80f3-7757faf68108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:39.012947 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-e8b20176-4116-4030-80f3-7757faf68108 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:39.014492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e8b20176-4116-4030-80f3-7757faf68108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:39.091051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11c703-664f-44d2-a2bc-60c0c538e9aa None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:39.098982 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 243/491] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:39.098982 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b20176-4116-4030-80f3-7757faf68108 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:39.130814 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8529a-a86d-410f-9517-456eaec824be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:39.131483 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8529a-a86d-410f-9517-456eaec824be None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:39.131762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8529a-a86d-410f-9517-456eaec824be None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:39.132046 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8529a-a86d-410f-9517-456eaec824be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:39.144457 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8529a-a86d-410f-9517-456eaec824be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:39.146034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8529a-a86d-410f-9517-456eaec824be None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:39.155207 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:39.161708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-b3d8529a-a86d-410f-9517-456eaec824be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:39.161708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:39.161708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:39.161708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:39.161708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:39.161708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:39.161708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:39.161708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:39.161708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:39.161708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:39.161708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:39.161708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:39.161708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:39.161708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:39.161708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:39.161708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:39.162835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:39.162835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:39.162835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:39.162835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:39.162835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:39.162835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:39.162835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:39.162835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:39.162835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:39.162835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:39.162835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:39.162835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:39.162835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:39.162835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:39.162835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:39.162835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:39.162835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:39.163931 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:39.163931 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:39.163931 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:39.163931 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 249/492] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:39.164464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e1ad2-66d1-4f1d-8fc5-de57bb1fecd1 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:39.164747 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e1ad2-66d1-4f1d-8fc5-de57bb1fecd1 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:39.165176 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e1ad2-66d1-4f1d-8fc5-de57bb1fecd1 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:39.166580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 250/493] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:39.169389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e1ad2-66d1-4f1d-8fc5-de57bb1fecd1 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:39.197378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:39.231907 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b20176-4116-4030-80f3-7757faf68108 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:39.275432 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1a423-bf3f-41df-a99f-21dae5d0b3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:39.276330 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1a423-bf3f-41df-a99f-21dae5d0b3bd None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:39.276787 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1a423-bf3f-41df-a99f-21dae5d0b3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:39.277205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1a423-bf3f-41df-a99f-21dae5d0b3bd None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:39.277995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1a423-bf3f-41df-a99f-21dae5d0b3bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:39.278540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1a423-bf3f-41df-a99f-21dae5d0b3bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:39.280366 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1a423-bf3f-41df-a99f-21dae5d0b3bd None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:39.294937 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 244/494] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:39.296472 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4e92f-cb77-486d-9a3d-262a85605477 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:39.296906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4e92f-cb77-486d-9a3d-262a85605477 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:39.297341 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4e92f-cb77-486d-9a3d-262a85605477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:39.312760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e1ad2-66d1-4f1d-8fc5-de57bb1fecd1 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:39.319489 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-17a4e92f-cb77-486d-9a3d-262a85605477 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:39.321005 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-17a4e92f-cb77-486d-9a3d-262a85605477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:39.350403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca38644-5529-465f-b0a4-d173c16dfef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:39.350970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca38644-5529-465f-b0a4-d173c16dfef2 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:39.351254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca38644-5529-465f-b0a4-d173c16dfef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:39.351525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca38644-5529-465f-b0a4-d173c16dfef2 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:39.352224 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca38644-5529-465f-b0a4-d173c16dfef2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:39.352530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca38644-5529-465f-b0a4-d173c16dfef2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:39.354016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca38644-5529-465f-b0a4-d173c16dfef2 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:39.371288 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 251/495] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:39.372253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca38644-5529-465f-b0a4-d173c16dfef2 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:39.399157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 245/496] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:39.400338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4e92f-cb77-486d-9a3d-262a85605477 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:39.432981 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:39.459166 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:39.483408 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca38644-5529-465f-b0a4-d173c16dfef2 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:39.523433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d5345-5bd8-4f2e-828f-07f2ae510605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:39.524249 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d5345-5bd8-4f2e-828f-07f2ae510605 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:39.524367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d5345-5bd8-4f2e-828f-07f2ae510605 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:39.524607 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d5345-5bd8-4f2e-828f-07f2ae510605 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:39.532237 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d5345-5bd8-4f2e-828f-07f2ae510605 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:39.533779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4e92f-cb77-486d-9a3d-262a85605477 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:39.533903 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d5345-5bd8-4f2e-828f-07f2ae510605 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:39.552136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-9e9d5345-5bd8-4f2e-828f-07f2ae510605 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 19:05:39.552136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:39.552136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:39.552136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:39.552136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:39.552136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:39.552136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:39.552136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:39.552136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:39.552136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:39.552136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:39.552136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:39.552136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:39.552136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:39.552136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:39.552136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:39.553347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:39.554430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:39.554430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:39.554430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:39.554430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:39.554430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:39.554430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:39.554430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:39.554430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:39.554430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 19:05:39.554430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:39.554430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 252/497] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:39.554430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d5345-5bd8-4f2e-828f-07f2ae510605 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:39.576656 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4b27a-c37c-42b8-9f17-0138a4c94ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:39.577520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4b27a-c37c-42b8-9f17-0138a4c94ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:39.578049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4b27a-c37c-42b8-9f17-0138a4c94ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:39.578462 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4b27a-c37c-42b8-9f17-0138a4c94ff1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:39.587598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4b27a-c37c-42b8-9f17-0138a4c94ff1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:39.589254 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4b27a-c37c-42b8-9f17-0138a4c94ff1 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:39.605205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-cac4b27a-c37c-42b8-9f17-0138a4c94ff1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:05:39.605205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:39.605205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:39.605205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:39.605205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:39.605205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:39.605205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:39.605205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:39.605205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:39.605205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:39.605205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:39.605205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:39.605205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:39.605205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:39.605205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:39.605205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:39.606348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:39.607531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:39.607531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:39.607531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:39.607531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:39.607531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:39.607531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:39.607531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:39.607531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:39.607531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:05:39.607531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:39.609423 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 246/498] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 31 19:05:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:39.610358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4b27a-c37c-42b8-9f17-0138a4c94ff1 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:39.684875 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d5345-5bd8-4f2e-828f-07f2ae510605 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:39.710130 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:39.719708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fa0db-7698-4dd0-b4ff-2f1920967943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:39.720350 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fa0db-7698-4dd0-b4ff-2f1920967943 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:39.720627 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fa0db-7698-4dd0-b4ff-2f1920967943 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:39.720896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fa0db-7698-4dd0-b4ff-2f1920967943 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:39.721924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4b27a-c37c-42b8-9f17-0138a4c94ff1 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:39.728411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:39.733569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545fa0db-7698-4dd0-b4ff-2f1920967943 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:39.738629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-545fa0db-7698-4dd0-b4ff-2f1920967943 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:39.738629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:39.738629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:39.738629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:39.738629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:39.738629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:39.738629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:39.738629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:39.738629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:39.738629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:39.738629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:39.738629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:39.738629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:39.738629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:39.738629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:39.738629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:39.739615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:39.739615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:39.739615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:39.739615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:39.739615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:39.739615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:39.739615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:39.739615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:39.739615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:39.739615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:39.739615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:39.739615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:39.739615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:39.739615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:39.739615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:39.739615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:39.740448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:39.740448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:39.740448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:39.740448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:39.740448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:39.740448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:39.740448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:39.740448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:39.740448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:39.740448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:39.740448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:39.740448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:39.740448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 253/499] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:39.741159 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fa0db-7698-4dd0-b4ff-2f1920967943 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:39.761142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d7b36-4eb2-4b6f-8a79-fc8eedebab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:39.761758 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d7b36-4eb2-4b6f-8a79-fc8eedebab31 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:39.762038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d7b36-4eb2-4b6f-8a79-fc8eedebab31 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:39.762314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d7b36-4eb2-4b6f-8a79-fc8eedebab31 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:39.763083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d7b36-4eb2-4b6f-8a79-fc8eedebab31 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:39.763392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d7b36-4eb2-4b6f-8a79-fc8eedebab31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:39.764845 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d7b36-4eb2-4b6f-8a79-fc8eedebab31 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:39.802970 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 247/500] 10.222.0.59 () {56 vars in 1137 bytes} [Wed Aug 31 19:05:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:39.804236 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b60111-e343-4a21-9f29-822ba5e0f340 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:39.804524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b60111-e343-4a21-9f29-822ba5e0f340 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:39.804797 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b60111-e343-4a21-9f29-822ba5e0f340 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:39.805995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 248/501] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:39.806684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b60111-e343-4a21-9f29-822ba5e0f340 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:39.852771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fa0db-7698-4dd0-b4ff-2f1920967943 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:39.887919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3d422-17bf-4707-abeb-34794d8b950d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:39.888449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3d422-17bf-4707-abeb-34794d8b950d None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:39.888727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3d422-17bf-4707-abeb-34794d8b950d None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:39.889034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3d422-17bf-4707-abeb-34794d8b950d None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:39.889757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3d422-17bf-4707-abeb-34794d8b950d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:39.890063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3d422-17bf-4707-abeb-34794d8b950d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:39.891536 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3d422-17bf-4707-abeb-34794d8b950d None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:39.914651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b60111-e343-4a21-9f29-822ba5e0f340 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:39.928306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 254/502] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 31 19:05:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:39.929145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3d422-17bf-4707-abeb-34794d8b950d None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:39.949210 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0996f3-7ca7-43be-837d-b8f748c050b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:39.949838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0996f3-7ca7-43be-837d-b8f748c050b7 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:39.950113 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0996f3-7ca7-43be-837d-b8f748c050b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:39.950381 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0996f3-7ca7-43be-837d-b8f748c050b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:39.961280 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0996f3-7ca7-43be-837d-b8f748c050b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:39.966059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-db0996f3-7ca7-43be-837d-b8f748c050b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:39.966059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:39.966059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:39.966059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:39.966059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:39.966059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:39.966059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:39.966059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:39.966059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:39.966059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:39.966059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:39.966059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:39.966059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:39.966059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:39.966059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:39.966059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:39.967124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:39.967124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:39.967124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:39.967124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:39.967124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:39.967124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:39.967124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:39.967124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:39.967124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:39.967124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:39.967124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:39.967124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:39.967124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:39.967124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:39.967124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:39.967124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:39.968002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:39.968002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:39.968002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:39.968002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:39.968002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:39.968002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:39.968002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:39.968002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:39.968002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:39.968002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:39.968002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:39.968002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:39.968002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 249/503] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:39.968695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0996f3-7ca7-43be-837d-b8f748c050b7 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:39.978673 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:39.987771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:40.032187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3d422-17bf-4707-abeb-34794d8b950d None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:40.067202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff80defb-f5cd-402d-b3a5-2d82aa7882c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:40.067809 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80defb-f5cd-402d-b3a5-2d82aa7882c8 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:40.068087 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80defb-f5cd-402d-b3a5-2d82aa7882c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:40.068357 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80defb-f5cd-402d-b3a5-2d82aa7882c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:40.068498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0996f3-7ca7-43be-837d-b8f748c050b7 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:40.069042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff80defb-f5cd-402d-b3a5-2d82aa7882c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:40.069345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff80defb-f5cd-402d-b3a5-2d82aa7882c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:40.070713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff80defb-f5cd-402d-b3a5-2d82aa7882c8 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:40.078718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 255/504] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:05:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:40.084585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263e3a8-8005-4df8-9bd5-702cba72ad14 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:40.084921 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263e3a8-8005-4df8-9bd5-702cba72ad14 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:40.085198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263e3a8-8005-4df8-9bd5-702cba72ad14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:40.103194 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-9263e3a8-8005-4df8-9bd5-702cba72ad14 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:40.104410 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-9263e3a8-8005-4df8-9bd5-702cba72ad14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:40.106022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26a566-04c7-4045-ae74-85f097249dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:40.106630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26a566-04c7-4045-ae74-85f097249dbc None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:40.106953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26a566-04c7-4045-ae74-85f097249dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:40.107219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26a566-04c7-4045-ae74-85f097249dbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:40.173534 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 256/505] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:40.174356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263e3a8-8005-4df8-9bd5-702cba72ad14 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:40.190593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26a566-04c7-4045-ae74-85f097249dbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:40.192842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26a566-04c7-4045-ae74-85f097249dbc None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:40.200793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:40.228805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:40.288393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263e3a8-8005-4df8-9bd5-702cba72ad14 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:40.302773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 250/506] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:40.304204 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26a566-04c7-4045-ae74-85f097249dbc None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:40.327762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077c408-d0c9-4c92-84d7-2887a6b0be4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:40.328379 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077c408-d0c9-4c92-84d7-2887a6b0be4c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:40.328657 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077c408-d0c9-4c92-84d7-2887a6b0be4c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:40.328927 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077c408-d0c9-4c92-84d7-2887a6b0be4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:40.409620 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f077c408-d0c9-4c92-84d7-2887a6b0be4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:40.412045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f077c408-d0c9-4c92-84d7-2887a6b0be4c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:40.415984 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26a566-04c7-4045-ae74-85f097249dbc None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:40.424209 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:40.461298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660578c-316b-43d2-92f8-500ee556f8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:40.463158 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660578c-316b-43d2-92f8-500ee556f8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:40.463625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660578c-316b-43d2-92f8-500ee556f8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:40.464078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660578c-316b-43d2-92f8-500ee556f8d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:40.465207 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4660578c-316b-43d2-92f8-500ee556f8d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:40.465735 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4660578c-316b-43d2-92f8-500ee556f8d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:40.468161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4660578c-316b-43d2-92f8-500ee556f8d9 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:40.478216 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:40.480779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 251/507] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:05:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:40.481997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660578c-316b-43d2-92f8-500ee556f8d9 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:40.541398 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 257/508] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:40.542252 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077c408-d0c9-4c92-84d7-2887a6b0be4c None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:40.591471 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660578c-316b-43d2-92f8-500ee556f8d9 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:40.624820 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2ab55-11a4-4d62-beb4-be88c0ace240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:40.625446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2ab55-11a4-4d62-beb4-be88c0ace240 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:40.625723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2ab55-11a4-4d62-beb4-be88c0ace240 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:40.625994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2ab55-11a4-4d62-beb4-be88c0ace240 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:40.636576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2ab55-11a4-4d62-beb4-be88c0ace240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:40.638031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2ab55-11a4-4d62-beb4-be88c0ace240 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:40.646742 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077c408-d0c9-4c92-84d7-2887a6b0be4c None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:40.652646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-9be2ab55-11a4-4d62-beb4-be88c0ace240 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:40.652646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:40.652646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:40.652646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:40.652646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:40.652646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:40.652646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:40.652646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:40.652646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:40.652646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:40.652646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:40.652646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:40.652646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:40.652646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:40.652646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:40.652646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:40.653838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:40.653838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:40.653838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:40.653838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:40.653838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:40.653838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:40.653838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:40.653838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:40.653838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:40.653838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:40.653838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:40.653838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:40.653838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:40.653838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:40.653838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:40.653838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:40.653838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:40.654969 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:40.654969 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:40.654969 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:40.658434 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 252/509] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:40.659368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2ab55-11a4-4d62-beb4-be88c0ace240 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:40.683383 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:40.683880 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f04043-efb6-47ce-b73c-4ca25d7780a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:40.684531 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f04043-efb6-47ce-b73c-4ca25d7780a9 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:40.684850 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f04043-efb6-47ce-b73c-4ca25d7780a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:40.685075 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f04043-efb6-47ce-b73c-4ca25d7780a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:40.696983 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f04043-efb6-47ce-b73c-4ca25d7780a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:40.699111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f04043-efb6-47ce-b73c-4ca25d7780a9 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:40.708635 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:40.717979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-37f04043-efb6-47ce-b73c-4ca25d7780a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:40.717979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:40.717979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:40.717979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:40.717979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:40.717979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:40.717979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:40.717979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:40.717979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:40.717979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:40.717979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:40.717979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:40.717979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:40.717979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:40.717979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:40.717979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:40.723427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:40.723427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:40.723427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:40.723427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:40.723427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:40.723427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:40.723427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:40.723427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:40.723427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:40.723427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:40.723427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:40.723427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:40.723427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:40.723427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:40.723427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:40.723427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:40.723427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:40.724322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:40.724322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:40.724322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:40.724322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 258/510] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:40.724322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f04043-efb6-47ce-b73c-4ca25d7780a9 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:40.775363 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2ab55-11a4-4d62-beb4-be88c0ace240 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:40.809020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec004817-e90f-40f1-8dd3-138e86b7d699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:40.809644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec004817-e90f-40f1-8dd3-138e86b7d699 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:40.809933 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec004817-e90f-40f1-8dd3-138e86b7d699 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:40.810211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec004817-e90f-40f1-8dd3-138e86b7d699 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:40.820825 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec004817-e90f-40f1-8dd3-138e86b7d699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:40.822337 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec004817-e90f-40f1-8dd3-138e86b7d699 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:40.822823 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f04043-efb6-47ce-b73c-4ca25d7780a9 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:40.837597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-ec004817-e90f-40f1-8dd3-138e86b7d699 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:40.837597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:40.837597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:40.837597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:40.837597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:40.837597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:40.837597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:40.837597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:40.837597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:40.837597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:40.837597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:40.837597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:40.837597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:40.837597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:40.837597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:40.837597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:40.838584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:40.838584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:40.838584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:40.838584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:40.838584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:40.838584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:40.838584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:40.838584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:40.838584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:40.838584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:40.838584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:40.838584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:40.838584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:40.838584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:40.838584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:40.838584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:40.838584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:40.839628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:40.839628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:40.839628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:40.839628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 253/511] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:40.840302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec004817-e90f-40f1-8dd3-138e86b7d699 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:40.860865 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3361717-71ea-4d4a-aa7c-855e7e5403fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:40.861509 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3361717-71ea-4d4a-aa7c-855e7e5403fa None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:40.861792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3361717-71ea-4d4a-aa7c-855e7e5403fa None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:40.862064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3361717-71ea-4d4a-aa7c-855e7e5403fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:40.873399 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3361717-71ea-4d4a-aa7c-855e7e5403fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:40.874849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3361717-71ea-4d4a-aa7c-855e7e5403fa None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:40.889426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-d3361717-71ea-4d4a-aa7c-855e7e5403fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:40.889426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:40.889426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:40.889426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:40.889426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:40.889426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:40.889426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:40.889426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:40.889426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:40.889426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:40.889426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:40.889426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:40.889426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:40.889426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:40.889426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:40.889426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:40.890475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:40.890475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:40.890475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:40.890475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:40.890475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:40.890475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:40.890475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:40.890475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:40.890475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:40.890475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:40.890475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:40.890475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:40.890475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:40.890475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:40.890475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:40.890475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:40.890475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:40.891743 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:40.891743 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:40.891743 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:40.891743 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 259/512] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:40.891743 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3361717-71ea-4d4a-aa7c-855e7e5403fa None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:40.913707 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:40.944133 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec004817-e90f-40f1-8dd3-138e86b7d699 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:40.945013 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:40.978387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d7a1a-bdd7-4859-8675-f80f81d6a685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:40.979060 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d7a1a-bdd7-4859-8675-f80f81d6a685 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:40.979342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d7a1a-bdd7-4859-8675-f80f81d6a685 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:40.979619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d7a1a-bdd7-4859-8675-f80f81d6a685 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:40.990430 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d7a1a-bdd7-4859-8675-f80f81d6a685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:40.991906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d7a1a-bdd7-4859-8675-f80f81d6a685 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:40.993180 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3361717-71ea-4d4a-aa7c-855e7e5403fa None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:41.006540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-179d7a1a-bdd7-4859-8675-f80f81d6a685 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:41.006540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:41.006540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:41.006540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:41.006540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:41.006540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:41.006540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:41.006540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:41.006540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:41.006540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:41.006540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:41.006540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:41.006540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:41.006540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:41.006540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:41.006540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:41.007568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:41.007568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:41.007568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:41.007568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:41.007568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:41.007568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:41.007568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:41.007568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:41.007568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:41.007568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:41.007568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:41.007568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:41.007568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:41.007568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:41.007568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:41.007568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:41.007568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:41.008483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:41.008483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:41.008483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:41.008483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 254/513] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:41.008483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d7a1a-bdd7-4859-8675-f80f81d6a685 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:41.032343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d57b6e-e539-47c6-a8bb-331f8c6ea9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:41.032958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d57b6e-e539-47c6-a8bb-331f8c6ea9b6 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:41.033239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d57b6e-e539-47c6-a8bb-331f8c6ea9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:41.033523 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d57b6e-e539-47c6-a8bb-331f8c6ea9b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:41.034156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d57b6e-e539-47c6-a8bb-331f8c6ea9b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:41.034458 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d57b6e-e539-47c6-a8bb-331f8c6ea9b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:41.035934 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d57b6e-e539-47c6-a8bb-331f8c6ea9b6 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:41.049781 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 260/514] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:41.051205 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d57b6e-e539-47c6-a8bb-331f8c6ea9b6 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:41.117675 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d7a1a-bdd7-4859-8675-f80f81d6a685 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:41.142913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:41.153344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b0377-486b-40dc-8ad1-a826b3fea7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:41.157016 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b0377-486b-40dc-8ad1-a826b3fea7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:41.157016 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b0377-486b-40dc-8ad1-a826b3fea7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:41.157016 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b0377-486b-40dc-8ad1-a826b3fea7b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:41.157016 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b0377-486b-40dc-8ad1-a826b3fea7b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:41.157016 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b0377-486b-40dc-8ad1-a826b3fea7b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:41.157396 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d57b6e-e539-47c6-a8bb-331f8c6ea9b6 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:41.157475 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b0377-486b-40dc-8ad1-a826b3fea7b6 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:41.169454 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 255/515] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:41.170284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b0377-486b-40dc-8ad1-a826b3fea7b6 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:41.172384 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:41.196875 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e51760-e487-4ec9-a1e9-b22bef523f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:41.197408 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e51760-e487-4ec9-a1e9-b22bef523f64 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:41.197693 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e51760-e487-4ec9-a1e9-b22bef523f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:41.197972 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e51760-e487-4ec9-a1e9-b22bef523f64 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:41.198623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e51760-e487-4ec9-a1e9-b22bef523f64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:41.198961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e51760-e487-4ec9-a1e9-b22bef523f64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:41.200350 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e51760-e487-4ec9-a1e9-b22bef523f64 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:41.212687 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 261/516] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:41.213545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e51760-e487-4ec9-a1e9-b22bef523f64 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:41.273376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b0377-486b-40dc-8ad1-a826b3fea7b6 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:41.307801 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffefa10-d601-4d30-834c-9f75de2393a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:41.308410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffefa10-d601-4d30-834c-9f75de2393a6 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:41.308694 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffefa10-d601-4d30-834c-9f75de2393a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:41.308970 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffefa10-d601-4d30-834c-9f75de2393a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:41.309619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffefa10-d601-4d30-834c-9f75de2393a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:41.310020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffefa10-d601-4d30-834c-9f75de2393a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:41.311462 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffefa10-d601-4d30-834c-9f75de2393a6 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:41.319478 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e51760-e487-4ec9-a1e9-b22bef523f64 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:41.323927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 256/517] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:41.324737 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffefa10-d601-4d30-834c-9f75de2393a6 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:41.354154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca11c7-66e9-4a12-b866-d7375552fdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:41.354764 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca11c7-66e9-4a12-b866-d7375552fdde None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:41.355077 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca11c7-66e9-4a12-b866-d7375552fdde None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:41.355356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca11c7-66e9-4a12-b866-d7375552fdde None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:41.356007 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca11c7-66e9-4a12-b866-d7375552fdde None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:41.356315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca11c7-66e9-4a12-b866-d7375552fdde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:41.357684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca11c7-66e9-4a12-b866-d7375552fdde None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:41.370107 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 262/518] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:41.370932 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca11c7-66e9-4a12-b866-d7375552fdde None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:41.372023 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:41.404606 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:41.429060 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffefa10-d601-4d30-834c-9f75de2393a6 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:41.474884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7320c4-c838-43d5-be29-850e42b28325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:41.475460 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7320c4-c838-43d5-be29-850e42b28325 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:41.475743 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7320c4-c838-43d5-be29-850e42b28325 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:41.476025 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7320c4-c838-43d5-be29-850e42b28325 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:41.483735 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7320c4-c838-43d5-be29-850e42b28325 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:41.484868 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca11c7-66e9-4a12-b866-d7375552fdde None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:41.485214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7320c4-c838-43d5-be29-850e42b28325 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:41.497906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-4d7320c4-c838-43d5-be29-850e42b28325 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:05:41.497906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:41.497906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:41.497906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:41.497906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:41.497906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:41.497906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:41.497906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:41.497906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:41.497906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:41.497906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:41.497906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:41.497906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:41.497906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:41.497906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:41.497906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:41.499083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:41.500389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:41.500389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:41.500389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:41.500389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:41.500389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:41.500389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:41.500389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:41.500389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:41.500389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:05:41.500389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:41.500389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 257/519] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:41.500389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7320c4-c838-43d5-be29-850e42b28325 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:41.525219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55cb63-2d11-43eb-8331-eed6fcae8f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:41.525855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55cb63-2d11-43eb-8331-eed6fcae8f1c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:41.526140 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55cb63-2d11-43eb-8331-eed6fcae8f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:41.526417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55cb63-2d11-43eb-8331-eed6fcae8f1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:41.533206 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55cb63-2d11-43eb-8331-eed6fcae8f1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:41.534635 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55cb63-2d11-43eb-8331-eed6fcae8f1c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:41.546981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-de55cb63-2d11-43eb-8331-eed6fcae8f1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:41.546981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:41.546981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:41.546981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:41.546981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:41.546981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:41.546981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:41.546981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:41.546981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:41.546981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:41.546981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:41.546981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:41.546981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:41.546981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:41.546981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:41.548045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:41.548045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:41.548045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:41.548045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:41.548045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:41.548045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:41.548045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:41.548045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:41.548045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:41.548045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:41.548045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:41.548045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:41.548045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:41.548045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:41.548045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:41.548045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:41.548045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:41.548947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:41.548947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:41.548947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:41.548947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:41.548947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:41.548947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:41.548947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:41.548947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:41.548947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 263/520] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:41.549427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55cb63-2d11-43eb-8331-eed6fcae8f1c None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:41.616175 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7320c4-c838-43d5-be29-850e42b28325 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:41.618472 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:41.645986 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:41.652303 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79940a7f-aec5-40cd-b619-1095287d035e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:41.652920 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79940a7f-aec5-40cd-b619-1095287d035e None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:41.653206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79940a7f-aec5-40cd-b619-1095287d035e None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:41.653484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79940a7f-aec5-40cd-b619-1095287d035e None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:41.654132 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79940a7f-aec5-40cd-b619-1095287d035e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:41.655238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55cb63-2d11-43eb-8331-eed6fcae8f1c None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:41.655683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79940a7f-aec5-40cd-b619-1095287d035e None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:41.688977 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f36fc-70a4-48a0-868e-8dcdcf212313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:41.689652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f36fc-70a4-48a0-868e-8dcdcf212313 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:41.689928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f36fc-70a4-48a0-868e-8dcdcf212313 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:41.690195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f36fc-70a4-48a0-868e-8dcdcf212313 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:41.704992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f36fc-70a4-48a0-868e-8dcdcf212313 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:41.708460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f36fc-70a4-48a0-868e-8dcdcf212313 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:41.729005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-424f36fc-70a4-48a0-868e-8dcdcf212313 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 19:05:41.729005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:41.729005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:41.729005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:41.729005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:41.729005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:41.729005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:41.729005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:41.729005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:41.729005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:41.729005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:41.729005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:41.729005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:41.729005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:41.729005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:41.729005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:41.734623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:41.735727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:41.735727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:41.735727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:41.735727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:41.735727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:41.735727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:41.735727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:41.735727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:41.735727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 19:05:41.735727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:41.735727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 264/521] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:41.735727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f36fc-70a4-48a0-868e-8dcdcf212313 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:41.746759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 258/522] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:05:41 2022] POST /identity/v3/users => generated 247 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:41.747626 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79940a7f-aec5-40cd-b619-1095287d035e None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:41.865560 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f36fc-70a4-48a0-868e-8dcdcf212313 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:41.876711 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79940a7f-aec5-40cd-b619-1095287d035e None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:41.905850 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af45f3-2c98-4504-828d-8813ec0b8844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:41.906494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af45f3-2c98-4504-828d-8813ec0b8844 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:41.906780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af45f3-2c98-4504-828d-8813ec0b8844 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:41.907090 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af45f3-2c98-4504-828d-8813ec0b8844 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:41.915524 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af45f3-2c98-4504-828d-8813ec0b8844 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:41.916586 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af45f3-2c98-4504-828d-8813ec0b8844 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:41.919495 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12ee86-7f6c-417f-9dbe-8f85384f56f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:41.920737 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12ee86-7f6c-417f-9dbe-8f85384f56f1 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:41.920737 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12ee86-7f6c-417f-9dbe-8f85384f56f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:41.920737 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12ee86-7f6c-417f-9dbe-8f85384f56f1 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:41.921372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12ee86-7f6c-417f-9dbe-8f85384f56f1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:41.921677 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12ee86-7f6c-417f-9dbe-8f85384f56f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:41.923132 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12ee86-7f6c-417f-9dbe-8f85384f56f1 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:41.929978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-63af45f3-2c98-4504-828d-8813ec0b8844 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:41.929978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:41.929978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:41.929978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:41.929978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:41.929978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:41.929978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:41.929978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:41.929978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:41.929978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:41.929978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:41.929978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:41.929978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:41.929978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:41.929978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:41.930863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:41.930863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:41.930863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:41.930863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:41.930863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:41.930863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:41.930863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:41.930863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:41.930863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:41.930863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:41.930863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:41.930863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:41.930863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:41.930863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:41.930863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:41.930863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:41.930863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:41.932060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:41.932060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:41.932060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:41.932060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:41.932060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:41.932060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:41.932060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:41.932060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:41.932060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 265/523] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:41.932539 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af45f3-2c98-4504-828d-8813ec0b8844 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:41.943659 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 259/524] 10.222.0.59 () {56 vars in 1137 bytes} [Wed Aug 31 19:05:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:41.944438 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12ee86-7f6c-417f-9dbe-8f85384f56f1 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:41.950456 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:41.965212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:42.044641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af45f3-2c98-4504-828d-8813ec0b8844 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:42.062913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12ee86-7f6c-417f-9dbe-8f85384f56f1 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:42.079502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61330c71-8156-46cc-9e61-690a4d995efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:42.080197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61330c71-8156-46cc-9e61-690a4d995efb None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:42.080477 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61330c71-8156-46cc-9e61-690a4d995efb None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:42.080746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61330c71-8156-46cc-9e61-690a4d995efb None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:42.081438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61330c71-8156-46cc-9e61-690a4d995efb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:42.082369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61330c71-8156-46cc-9e61-690a4d995efb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:42.084475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61330c71-8156-46cc-9e61-690a4d995efb None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:42.102757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8283bb-3cee-431b-8168-2cc994c6ba81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:42.103240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 266/525] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:42.103791 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8283bb-3cee-431b-8168-2cc994c6ba81 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:42.104076 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61330c71-8156-46cc-9e61-690a4d995efb None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:42.104389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8283bb-3cee-431b-8168-2cc994c6ba81 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:42.104829 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8283bb-3cee-431b-8168-2cc994c6ba81 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:42.105600 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8283bb-3cee-431b-8168-2cc994c6ba81 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:42.105911 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8283bb-3cee-431b-8168-2cc994c6ba81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:42.107377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8283bb-3cee-431b-8168-2cc994c6ba81 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:42.130598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 260/526] 10.222.0.59 () {56 vars in 1139 bytes} [Wed Aug 31 19:05:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:42.131531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8283bb-3cee-431b-8168-2cc994c6ba81 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:42.196362 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:42.220730 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:42.227149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61330c71-8156-46cc-9e61-690a4d995efb None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:42.248785 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8283bb-3cee-431b-8168-2cc994c6ba81 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:42.268593 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65228bc-16cc-422f-9028-a073c6831dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:42.269771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65228bc-16cc-422f-9028-a073c6831dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:42.270265 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65228bc-16cc-422f-9028-a073c6831dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:42.270742 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65228bc-16cc-422f-9028-a073c6831dc4 None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:42.271649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65228bc-16cc-422f-9028-a073c6831dc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:42.272153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65228bc-16cc-422f-9028-a073c6831dc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:42.273713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65228bc-16cc-422f-9028-a073c6831dc4 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:42.294463 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 267/527] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:42.294463 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65228bc-16cc-422f-9028-a073c6831dc4 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:42.301896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a316bf-ef33-45b9-975a-70c654ea9688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:42.302537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a316bf-ef33-45b9-975a-70c654ea9688 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:42.302901 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a316bf-ef33-45b9-975a-70c654ea9688 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:42.303263 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a316bf-ef33-45b9-975a-70c654ea9688 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:42.377569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a316bf-ef33-45b9-975a-70c654ea9688 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:42.379611 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a316bf-ef33-45b9-975a-70c654ea9688 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:42.409808 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65228bc-16cc-422f-9028-a073c6831dc4 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:42.443156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:42.450613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdfb06-a1ea-4cb1-bebe-e9e2bee76cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:42.451269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdfb06-a1ea-4cb1-bebe-e9e2bee76cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:42.451852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdfb06-a1ea-4cb1-bebe-e9e2bee76cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:42.452131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdfb06-a1ea-4cb1-bebe-e9e2bee76cb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:42.453070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdfb06-a1ea-4cb1-bebe-e9e2bee76cb1 None admin] RBAC: Adding query filter params (role.id=a9ef0098e982437fb0127827fa2cad38, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=f281d778f1404984b4d778a1c26aef1d) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:42.453391 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdfb06-a1ea-4cb1-bebe-e9e2bee76cb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:42.454857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdfb06-a1ea-4cb1-bebe-e9e2bee76cb1 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:42.466895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:42.467093 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 268/528] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:42 2022] GET /identity/v3/role_assignments?user.id=f281d778f1404984b4d778a1c26aef1d&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=a9ef0098e982437fb0127827fa2cad38 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:42.467093 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdfb06-a1ea-4cb1-bebe-e9e2bee76cb1 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:42.505489 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 261/529] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:42.507795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a316bf-ef33-45b9-975a-70c654ea9688 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:42.586472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdfb06-a1ea-4cb1-bebe-e9e2bee76cb1 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:42.635616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791a419-19ff-4f62-b07b-889efd9b8c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:42.636332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791a419-19ff-4f62-b07b-889efd9b8c1d None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:42.636677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791a419-19ff-4f62-b07b-889efd9b8c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:42.637019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791a419-19ff-4f62-b07b-889efd9b8c1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:42.658730 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a316bf-ef33-45b9-975a-70c654ea9688 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:42.717517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32bf69-6a9e-4bb7-8614-44523efa31b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:42.718812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32bf69-6a9e-4bb7-8614-44523efa31b7 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:42.719549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32bf69-6a9e-4bb7-8614-44523efa31b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:42.720007 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32bf69-6a9e-4bb7-8614-44523efa31b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:42.720824 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:42.722821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32bf69-6a9e-4bb7-8614-44523efa31b7 None admin] RBAC: Adding query filter params (role.id=a9ef0098e982437fb0127827fa2cad38, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=d461422ce98442bcbe6b254a8dad5e8a) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:42.723335 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32bf69-6a9e-4bb7-8614-44523efa31b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:42.725268 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32bf69-6a9e-4bb7-8614-44523efa31b7 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:42.729774 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791a419-19ff-4f62-b07b-889efd9b8c1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:42.731393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791a419-19ff-4f62-b07b-889efd9b8c1d None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:42.739326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 262/530] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:42 2022] GET /identity/v3/role_assignments?user.id=d461422ce98442bcbe6b254a8dad5e8a&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=a9ef0098e982437fb0127827fa2cad38 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:42.739326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32bf69-6a9e-4bb7-8614-44523efa31b7 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:42.815679 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:42.892222 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32bf69-6a9e-4bb7-8614-44523efa31b7 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:42.936742 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e37d17-ddc5-46b7-aef2-cf24c317bffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:42.937394 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e37d17-ddc5-46b7-aef2-cf24c317bffc None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:42.937676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e37d17-ddc5-46b7-aef2-cf24c317bffc None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:42.937952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e37d17-ddc5-46b7-aef2-cf24c317bffc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:42.952246 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e37d17-ddc5-46b7-aef2-cf24c317bffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:42.953759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e37d17-ddc5-46b7-aef2-cf24c317bffc None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:42.960471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 269/531] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:42.970515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791a419-19ff-4f62-b07b-889efd9b8c1d None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:42.978500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-e9e37d17-ddc5-46b7-aef2-cf24c317bffc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:42.978500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:42.978500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:42.978500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:42.978500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:42.978500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:42.978500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:42.978500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:42.978500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:42.978500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:42.978500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:42.978500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:42.978500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:42.978500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:42.978500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:42.978500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:42.979690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:42.979690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:42.979690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:42.979690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:42.979690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:42.979690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:42.979690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:42.979690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:42.979690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:42.979690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:42.979690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:42.979690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:42.979690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:42.979690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:42.979690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:42.979690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:42.979690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:42.980768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:42.980768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:42.980768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:42.980768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 263/532] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:42.986954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e37d17-ddc5-46b7-aef2-cf24c317bffc None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:43.110812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:43.124008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e37d17-ddc5-46b7-aef2-cf24c317bffc None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:43.179029 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:43.185945 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791a419-19ff-4f62-b07b-889efd9b8c1d None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:43.199579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6836921-f1bc-4133-9394-7fd6964962e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:43.200207 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836921-f1bc-4133-9394-7fd6964962e5 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:43.200558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836921-f1bc-4133-9394-7fd6964962e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:43.200752 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836921-f1bc-4133-9394-7fd6964962e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:43.201408 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6836921-f1bc-4133-9394-7fd6964962e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:43.201708 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6836921-f1bc-4133-9394-7fd6964962e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:43.203118 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6836921-f1bc-4133-9394-7fd6964962e5 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:43.226919 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 264/533] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:43.235915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6836921-f1bc-4133-9394-7fd6964962e5 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:43.236858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a194e2-5d5d-4c90-8c1b-fa34762ea2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:43.238297 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a194e2-5d5d-4c90-8c1b-fa34762ea2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:43.238771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a194e2-5d5d-4c90-8c1b-fa34762ea2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:43.239483 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a194e2-5d5d-4c90-8c1b-fa34762ea2f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:43.262578 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a194e2-5d5d-4c90-8c1b-fa34762ea2f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:43.266626 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a194e2-5d5d-4c90-8c1b-fa34762ea2f1 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:43.297821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-b0a194e2-5d5d-4c90-8c1b-fa34762ea2f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:43.297821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:43.297821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:43.297821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:43.297821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:43.297821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:43.297821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:43.297821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:43.297821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:43.297821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:43.297821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:43.297821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:43.297821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:43.297821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:43.297821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:43.297821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:43.299086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:43.299086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:43.299086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:43.299086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:43.299086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:43.299086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:43.299086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:43.299086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:43.299086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:43.299086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:43.299086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:43.299086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:43.299086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:43.299086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:43.299086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:43.299086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:43.299086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:43.299961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:43.299961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:43.299961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:43.299961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 270/534] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:43.307379 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a194e2-5d5d-4c90-8c1b-fa34762ea2f1 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:43.428134 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6836921-f1bc-4133-9394-7fd6964962e5 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:43.495730 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125165e-6d2b-4a4e-b37f-d44cd7bc1e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:43.496292 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125165e-6d2b-4a4e-b37f-d44cd7bc1e0a None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:43.496573 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125165e-6d2b-4a4e-b37f-d44cd7bc1e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:43.496846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125165e-6d2b-4a4e-b37f-d44cd7bc1e0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:43.509201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125165e-6d2b-4a4e-b37f-d44cd7bc1e0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:43.510667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125165e-6d2b-4a4e-b37f-d44cd7bc1e0a None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:43.513192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a194e2-5d5d-4c90-8c1b-fa34762ea2f1 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:43.539924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-3125165e-6d2b-4a4e-b37f-d44cd7bc1e0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:43.539924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:43.539924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:43.539924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:43.539924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:43.539924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:43.539924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:43.539924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:43.539924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:43.539924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:43.539924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:43.539924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:43.539924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:43.539924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:43.539924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:43.540862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:43.540862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:43.540862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:43.540862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:43.540862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:43.540862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:43.540862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:43.540862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:43.540862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:43.540862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:43.540862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:43.540862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:43.540862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:43.540862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:43.540862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:43.540862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:43.540862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:43.541870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:43.541870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:43.541870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:43.541870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:43.541870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:43.541870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:43.541870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:43.541870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:43.542348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 265/535] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:43.552578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125165e-6d2b-4a4e-b37f-d44cd7bc1e0a None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:43.577415 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:43.592018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a089bb-04d2-48ce-8340-40171079b699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:43.592912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a089bb-04d2-48ce-8340-40171079b699 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:43.597318 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a089bb-04d2-48ce-8340-40171079b699 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:43.597318 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a089bb-04d2-48ce-8340-40171079b699 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:43.597318 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a089bb-04d2-48ce-8340-40171079b699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:43.597318 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a089bb-04d2-48ce-8340-40171079b699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:43.597318 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a089bb-04d2-48ce-8340-40171079b699 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:43.610751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 271/536] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:43.618157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a089bb-04d2-48ce-8340-40171079b699 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:43.635225 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:43.748255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125165e-6d2b-4a4e-b37f-d44cd7bc1e0a None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:43.816708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a089bb-04d2-48ce-8340-40171079b699 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:43.825249 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6951e44-9d31-4044-928b-8c6fa99c7e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:43.825869 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6951e44-9d31-4044-928b-8c6fa99c7e2f None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:43.826144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6951e44-9d31-4044-928b-8c6fa99c7e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:43.826410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6951e44-9d31-4044-928b-8c6fa99c7e2f None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:43.827146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6951e44-9d31-4044-928b-8c6fa99c7e2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:43.827437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6951e44-9d31-4044-928b-8c6fa99c7e2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:43.828883 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6951e44-9d31-4044-928b-8c6fa99c7e2f None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:43.852468 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 266/537] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:43.853813 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d486a-64fe-4ed7-be5c-65be1c12400b None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:43.854107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d486a-64fe-4ed7-be5c-65be1c12400b None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:43.854379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d486a-64fe-4ed7-be5c-65be1c12400b None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:43.859577 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 267/538] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:43.860797 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d486a-64fe-4ed7-be5c-65be1c12400b None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:43.877664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579ab89-1f82-4b0c-87d6-b4b2af02f2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:43.879317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579ab89-1f82-4b0c-87d6-b4b2af02f2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:43.879920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579ab89-1f82-4b0c-87d6-b4b2af02f2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:43.880200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579ab89-1f82-4b0c-87d6-b4b2af02f2f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:43.889353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579ab89-1f82-4b0c-87d6-b4b2af02f2f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:43.891279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579ab89-1f82-4b0c-87d6-b4b2af02f2f7 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:43.906533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-5579ab89-1f82-4b0c-87d6-b4b2af02f2f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:43.906533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:43.906533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:43.906533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:43.906533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:43.906533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:43.906533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:43.906533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:43.906533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:43.906533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:43.906533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:43.906533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:43.906533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:43.906533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:43.906533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:43.907507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:43.907507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:43.907507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:43.907507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:43.907507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:43.907507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:43.907507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:43.907507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:43.907507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:43.907507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:43.907507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:43.907507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:43.907507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:43.907507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:43.907507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:43.907507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:43.907507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:43.908385 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:43.908385 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:43.908385 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:43.908385 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:43.908385 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:43.908385 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:43.908385 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:43.908385 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:43.911274 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 272/539] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:43.912848 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91348d-20c7-4024-8f4b-d5a63adcfa4c None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:43.913291 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91348d-20c7-4024-8f4b-d5a63adcfa4c None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:43.913638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91348d-20c7-4024-8f4b-d5a63adcfa4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:43.932967 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-bc91348d-20c7-4024-8f4b-d5a63adcfa4c None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:43.934395 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-bc91348d-20c7-4024-8f4b-d5a63adcfa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:43.943022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:43.974547 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d486a-64fe-4ed7-be5c-65be1c12400b None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:44.004472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:44.010071 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 273/540] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:44.010920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91348d-20c7-4024-8f4b-d5a63adcfa4c None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:44.026808 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58477d-c03a-4857-bbda-25d865e5209b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:44.027704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58477d-c03a-4857-bbda-25d865e5209b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:44.027990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58477d-c03a-4857-bbda-25d865e5209b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:44.028266 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58477d-c03a-4857-bbda-25d865e5209b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:44.029026 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58477d-c03a-4857-bbda-25d865e5209b None admin] RBAC: Adding query filter params (role.id=a9ef0098e982437fb0127827fa2cad38, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=108a698aa3e6477b9ae896d68f22bd0c) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:44.029342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58477d-c03a-4857-bbda-25d865e5209b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:44.030987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58477d-c03a-4857-bbda-25d865e5209b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:44.042616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 268/541] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:43 2022] GET /identity/v3/role_assignments?user.id=108a698aa3e6477b9ae896d68f22bd0c&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=a9ef0098e982437fb0127827fa2cad38 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:44.143892 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91348d-20c7-4024-8f4b-d5a63adcfa4c None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:44.187833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba5cd8-62bd-437c-94a7-3c64c7c698fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:44.188525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba5cd8-62bd-437c-94a7-3c64c7c698fd None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:44.188809 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba5cd8-62bd-437c-94a7-3c64c7c698fd None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:44.189080 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba5cd8-62bd-437c-94a7-3c64c7c698fd None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:44.189795 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba5cd8-62bd-437c-94a7-3c64c7c698fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:44.190098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba5cd8-62bd-437c-94a7-3c64c7c698fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:44.191618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba5cd8-62bd-437c-94a7-3c64c7c698fd None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:44.208477 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 274/542] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:44.214416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58477d-c03a-4857-bbda-25d865e5209b None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:44.376915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58477d-c03a-4857-bbda-25d865e5209b None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:44.401205 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba5cd8-62bd-437c-94a7-3c64c7c698fd None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:44.429106 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10bf77-a36e-4dd6-a9ee-ddff0e709e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:44.429728 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10bf77-a36e-4dd6-a9ee-ddff0e709e2b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:44.430007 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10bf77-a36e-4dd6-a9ee-ddff0e709e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:44.430275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10bf77-a36e-4dd6-a9ee-ddff0e709e2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:44.431034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10bf77-a36e-4dd6-a9ee-ddff0e709e2b None admin] RBAC: Adding query filter params (role.id=a9ef0098e982437fb0127827fa2cad38, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=b7a51545146846cc8284b7044b32198a) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:44.431344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10bf77-a36e-4dd6-a9ee-ddff0e709e2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:44.432778 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10bf77-a36e-4dd6-a9ee-ddff0e709e2b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:44.437306 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:44.442248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 269/543] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:44 2022] GET /identity/v3/role_assignments?user.id=b7a51545146846cc8284b7044b32198a&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=a9ef0098e982437fb0127827fa2cad38 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:44.514026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:44.578018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba5cd8-62bd-437c-94a7-3c64c7c698fd None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:44.620246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0adcea-2b59-48af-8cb3-1590cfa20834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:44.620886 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0adcea-2b59-48af-8cb3-1590cfa20834 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:44.621162 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0adcea-2b59-48af-8cb3-1590cfa20834 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:44.621430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0adcea-2b59-48af-8cb3-1590cfa20834 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:44.633093 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0adcea-2b59-48af-8cb3-1590cfa20834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:44.638572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-0c0adcea-2b59-48af-8cb3-1590cfa20834 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:44.638572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:44.638572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:44.638572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:44.638572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:44.638572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:44.638572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:44.638572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:44.638572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:44.638572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:44.638572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:44.638572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:44.638572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:44.638572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:44.638572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:44.638572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:44.639584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:44.639584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:44.639584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:44.639584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:44.639584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:44.639584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:44.639584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:44.639584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:44.639584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:44.639584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:44.639584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:44.639584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:44.639584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:44.639584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:44.639584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:44.639584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:44.640658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:44.640658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:44.640658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:44.640658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:44.640658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:44.640658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:44.640658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:44.640658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:44.640658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:44.640658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:44.640658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:44.640658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:44.640658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 275/544] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:44.648235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10bf77-a36e-4dd6-a9ee-ddff0e709e2b None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:44.791882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10bf77-a36e-4dd6-a9ee-ddff0e709e2b None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:44.837943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355a234-834e-4dde-8531-dde3b154d4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:44.838477 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355a234-834e-4dde-8531-dde3b154d4de None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:44.838753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355a234-834e-4dde-8531-dde3b154d4de None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:44.839054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355a234-834e-4dde-8531-dde3b154d4de None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:44.839794 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355a234-834e-4dde-8531-dde3b154d4de None admin] RBAC: Adding query filter params (name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:44.840096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355a234-834e-4dde-8531-dde3b154d4de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:44.841460 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355a234-834e-4dde-8531-dde3b154d4de None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:44.856206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 270/545] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:05:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:44.861520 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0adcea-2b59-48af-8cb3-1590cfa20834 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:44.951938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fedc2-5375-43e2-9eb2-01cb04c846cd None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:44.952480 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fedc2-5375-43e2-9eb2-01cb04c846cd None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:44.952480 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fedc2-5375-43e2-9eb2-01cb04c846cd None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:44.953824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 271/546] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:44.966605 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b1d3c-511c-4f65-ab3e-3c39ec6f6f59 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:44.967228 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b1d3c-511c-4f65-ab3e-3c39ec6f6f59 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:44.967610 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b1d3c-511c-4f65-ab3e-3c39ec6f6f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:44.999117 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0adcea-2b59-48af-8cb3-1590cfa20834 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:45.004721 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-211b1d3c-511c-4f65-ab3e-3c39ec6f6f59 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:45.006225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-211b1d3c-511c-4f65-ab3e-3c39ec6f6f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:45.031256 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:45.036868 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84831e-4451-4343-a54f-3956e9bcf0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:45.037707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84831e-4451-4343-a54f-3956e9bcf0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:45.037991 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84831e-4451-4343-a54f-3956e9bcf0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:45.038270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84831e-4451-4343-a54f-3956e9bcf0a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:45.054169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84831e-4451-4343-a54f-3956e9bcf0a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:45.056024 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84831e-4451-4343-a54f-3956e9bcf0a5 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:45.073352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:45.076982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-ac84831e-4451-4343-a54f-3956e9bcf0a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:45.076982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:45.076982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:45.076982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:45.076982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:45.076982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:45.076982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:45.076982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:45.076982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:45.076982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:45.076982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:45.076982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:45.076982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:45.076982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:45.076982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:45.076982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:45.077950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:45.077950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:45.077950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:45.077950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:45.077950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:45.077950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:45.077950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:45.077950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:45.077950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:45.077950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:45.077950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:45.077950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:45.077950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:45.077950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:45.077950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:45.077950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:45.077950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:45.078817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:45.078817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:45.078817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:45.083007 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 276/547] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:45.088041 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26694e-e352-46ee-9c69-17e16761ccf2 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:45.088041 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26694e-e352-46ee-9c69-17e16761ccf2 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:45.088041 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26694e-e352-46ee-9c69-17e16761ccf2 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:45.088041 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 277/548] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:45.088041 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26694e-e352-46ee-9c69-17e16761ccf2 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:45.096014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 272/549] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:45.098159 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ea73d-38b6-48c5-9c0c-4ab63b3dd871 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:45.098685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ea73d-38b6-48c5-9c0c-4ab63b3dd871 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:45.099185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ea73d-38b6-48c5-9c0c-4ab63b3dd871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:45.124350 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d33ea73d-38b6-48c5-9c0c-4ab63b3dd871 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:45.126733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d33ea73d-38b6-48c5-9c0c-4ab63b3dd871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:45.202217 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26694e-e352-46ee-9c69-17e16761ccf2 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:45.213611 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 273/550] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:45.265089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d5cac-3a80-431f-bfc5-0aed35c46e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:45.265728 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d5cac-3a80-431f-bfc5-0aed35c46e1f None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:45.266012 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d5cac-3a80-431f-bfc5-0aed35c46e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:45.266285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d5cac-3a80-431f-bfc5-0aed35c46e1f None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:45.266972 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d5cac-3a80-431f-bfc5-0aed35c46e1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:45.267283 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d5cac-3a80-431f-bfc5-0aed35c46e1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:45.268680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d5cac-3a80-431f-bfc5-0aed35c46e1f None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:45.294521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 278/551] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:45.300167 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ea73d-38b6-48c5-9c0c-4ab63b3dd871 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:45.442344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ea73d-38b6-48c5-9c0c-4ab63b3dd871 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:45.467455 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:45.479856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f720a-37dd-4d78-b930-dc5749a50551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:45.484502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f720a-37dd-4d78-b930-dc5749a50551 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:45.484502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f720a-37dd-4d78-b930-dc5749a50551 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:45.484502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f720a-37dd-4d78-b930-dc5749a50551 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:45.489209 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f720a-37dd-4d78-b930-dc5749a50551 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:45.490689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f720a-37dd-4d78-b930-dc5749a50551 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:45.504598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-123f720a-37dd-4d78-b930-dc5749a50551 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:05:45.504598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:45.504598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:45.504598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:45.504598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:45.504598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:45.504598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:45.504598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:45.504598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:45.504598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:45.504598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:45.504598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:45.504598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:45.504598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:45.504598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:45.504598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:45.505590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:45.506531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:45.506531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:45.506531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:45.506531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:45.506531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:45.506531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:45.506531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:45.506531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:45.506531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:05:45.506531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:45.507920 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 274/552] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:05:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:45.512351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d5cac-3a80-431f-bfc5-0aed35c46e1f None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:45.560647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f720a-37dd-4d78-b930-dc5749a50551 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:45.570941 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:45.660802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d5cac-3a80-431f-bfc5-0aed35c46e1f None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:45.689970 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f720a-37dd-4d78-b930-dc5749a50551 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:45.709035 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3289dcc-7694-4f3c-ab5c-d807c97db637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:45.709583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3289dcc-7694-4f3c-ab5c-d807c97db637 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:45.709865 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3289dcc-7694-4f3c-ab5c-d807c97db637 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:45.710141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3289dcc-7694-4f3c-ab5c-d807c97db637 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:45.711197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3289dcc-7694-4f3c-ab5c-d807c97db637 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:45.711508 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3289dcc-7694-4f3c-ab5c-d807c97db637 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:45.713117 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3289dcc-7694-4f3c-ab5c-d807c97db637 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:45.733466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e102f-d915-487c-938b-035ccec47e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:45.734476 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e102f-d915-487c-938b-035ccec47e2e None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:45.735140 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e102f-d915-487c-938b-035ccec47e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:45.735669 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e102f-d915-487c-938b-035ccec47e2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:45.739694 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 279/553] 10.222.0.59 () {56 vars in 1135 bytes} [Wed Aug 31 19:05:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:45.740672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3289dcc-7694-4f3c-ab5c-d807c97db637 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:45.750550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e102f-d915-487c-938b-035ccec47e2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:45.756854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-444e102f-d915-487c-938b-035ccec47e2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:45.756854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:45.756854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:45.756854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:45.756854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:45.756854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:45.756854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:45.756854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:45.756854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:45.756854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:45.756854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:45.756854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:45.756854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:45.756854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:45.756854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:45.756854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:45.757887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:45.757887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:45.757887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:45.757887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:45.757887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:45.757887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:45.757887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:45.757887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:45.757887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:45.757887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:45.757887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:45.757887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:45.757887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:45.757887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:45.757887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:45.757887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:45.758732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:45.758732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:45.758732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:45.758732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:45.758732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:45.758732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:45.758732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:45.758732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:45.758732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:45.758732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:45.758732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:45.758732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:45.762988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 275/554] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:45.763979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e102f-d915-487c-938b-035ccec47e2e None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:45.820705 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:45.876258 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:45.890540 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3289dcc-7694-4f3c-ab5c-d807c97db637 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:45.903479 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e102f-d915-487c-938b-035ccec47e2e None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:45.938060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a23086-ebdd-4943-81da-f1d0e9e43aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:45.939283 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a23086-ebdd-4943-81da-f1d0e9e43aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:45.940008 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a23086-ebdd-4943-81da-f1d0e9e43aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:45.941392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a23086-ebdd-4943-81da-f1d0e9e43aa8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:45.950461 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214baad-bda0-4973-9f1d-307c489f0f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:45.951132 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214baad-bda0-4973-9f1d-307c489f0f61 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:45.951429 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214baad-bda0-4973-9f1d-307c489f0f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:45.951664 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214baad-bda0-4973-9f1d-307c489f0f61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:45.957404 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a23086-ebdd-4943-81da-f1d0e9e43aa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:45.963887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-e3a23086-ebdd-4943-81da-f1d0e9e43aa8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:45.963887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:45.963887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:45.963887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:45.963887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:45.963887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:45.963887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:45.963887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:45.963887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:45.963887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:45.963887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:45.963887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:45.963887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:45.963887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:45.963887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:45.963887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:45.964820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:45.964820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:45.964820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:45.964820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:45.964820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:45.964820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:45.964820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:45.964820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:45.964820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:45.964820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:45.964820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:45.964820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:45.964820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:45.964820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:45.964820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:45.964820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:45.965668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:45.965668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:45.965668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:45.965668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:45.965668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:45.965668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:45.965668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:45.965668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:45.965668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:45.965668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:45.965668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:45.965668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:45.971008 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 280/555] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:45.971008 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a23086-ebdd-4943-81da-f1d0e9e43aa8 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:46.054687 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214baad-bda0-4973-9f1d-307c489f0f61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:46.056789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214baad-bda0-4973-9f1d-307c489f0f61 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:46.092912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a23086-ebdd-4943-81da-f1d0e9e43aa8 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:46.113217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:46.136501 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d389b-c2bf-4f2b-a6ec-3d0cae9311fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:46.137382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d389b-c2bf-4f2b-a6ec-3d0cae9311fd None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:46.137904 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d389b-c2bf-4f2b-a6ec-3d0cae9311fd None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:46.138569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d389b-c2bf-4f2b-a6ec-3d0cae9311fd None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:46.139430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d389b-c2bf-4f2b-a6ec-3d0cae9311fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:46.139937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d389b-c2bf-4f2b-a6ec-3d0cae9311fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:46.141788 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d389b-c2bf-4f2b-a6ec-3d0cae9311fd None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:46.157156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 281/556] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:05:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:46.158459 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d389b-c2bf-4f2b-a6ec-3d0cae9311fd None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:46.173316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:46.194925 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 276/557] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:46.194925 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214baad-bda0-4973-9f1d-307c489f0f61 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:46.303137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d389b-c2bf-4f2b-a6ec-3d0cae9311fd None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:46.338010 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214baad-bda0-4973-9f1d-307c489f0f61 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:46.342067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e86040-7c8a-45e8-818d-4180dc75edb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:46.342935 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e86040-7c8a-45e8-818d-4180dc75edb3 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:46.343954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e86040-7c8a-45e8-818d-4180dc75edb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:46.344674 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e86040-7c8a-45e8-818d-4180dc75edb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:46.345547 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e86040-7c8a-45e8-818d-4180dc75edb3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:46.346076 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e86040-7c8a-45e8-818d-4180dc75edb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:46.347765 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e86040-7c8a-45e8-818d-4180dc75edb3 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:46.357210 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 282/558] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:05:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:46.358454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e86040-7c8a-45e8-818d-4180dc75edb3 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:46.386664 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998cd46-2453-4da2-89af-9cf94aa727c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:46.387328 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998cd46-2453-4da2-89af-9cf94aa727c1 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:46.387600 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998cd46-2453-4da2-89af-9cf94aa727c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:46.387867 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998cd46-2453-4da2-89af-9cf94aa727c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:46.399039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998cd46-2453-4da2-89af-9cf94aa727c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:46.400485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998cd46-2453-4da2-89af-9cf94aa727c1 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:46.403342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:46.419235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-e998cd46-2453-4da2-89af-9cf94aa727c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:46.419235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:46.419235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:46.419235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:46.419235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:46.419235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:46.419235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:46.419235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:46.419235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:46.419235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:46.419235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:46.419235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:46.419235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:46.419235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:46.419235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:46.419235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:46.420325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:46.420325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:46.420325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:46.420325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:46.420325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:46.420325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:46.420325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:46.420325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:46.420325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:46.420325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:46.420325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:46.420325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:46.420325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:46.420325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:46.420325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:46.420325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:46.420325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:46.421283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:46.421283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:46.421283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:46.421283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 277/559] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:46.421997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3c3d1-1036-4381-87c0-5474f8783fcb None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:46.422179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3c3d1-1036-4381-87c0-5474f8783fcb None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:46.422414 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3c3d1-1036-4381-87c0-5474f8783fcb None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:46.426929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 278/560] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:46.426929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3c3d1-1036-4381-87c0-5474f8783fcb None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:46.448637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:46.487960 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e86040-7c8a-45e8-818d-4180dc75edb3 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:46.531396 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff03d5e-bd76-439f-a4f9-fa1a5a3b995f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:46.532040 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff03d5e-bd76-439f-a4f9-fa1a5a3b995f None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:46.532333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff03d5e-bd76-439f-a4f9-fa1a5a3b995f None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:46.532598 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff03d5e-bd76-439f-a4f9-fa1a5a3b995f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:46.545134 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff03d5e-bd76-439f-a4f9-fa1a5a3b995f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:46.546695 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff03d5e-bd76-439f-a4f9-fa1a5a3b995f None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:46.548616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3c3d1-1036-4381-87c0-5474f8783fcb None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:46.562433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-2ff03d5e-bd76-439f-a4f9-fa1a5a3b995f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:46.562433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:46.562433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:46.562433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:46.562433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:46.562433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:46.562433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:46.562433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:46.562433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:46.562433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:46.562433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:46.562433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:46.562433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:46.562433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:46.562433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:46.562433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:46.564141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:46.564141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:46.564141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:46.564141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:46.564141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:46.564141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:46.564141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:46.564141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:46.564141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:46.564141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:46.564141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:46.564141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:46.564141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:46.564141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:46.564141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:46.564141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:46.564141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:46.565805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:46.565805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:46.565805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:46.565805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 283/561] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:46.565805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff03d5e-bd76-439f-a4f9-fa1a5a3b995f None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:46.591000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df0c134-fcb4-478e-a3de-44fb282dce9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:46.591644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0c134-fcb4-478e-a3de-44fb282dce9e None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:46.591914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0c134-fcb4-478e-a3de-44fb282dce9e None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:46.592186 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0c134-fcb4-478e-a3de-44fb282dce9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:46.604016 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df0c134-fcb4-478e-a3de-44fb282dce9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:46.605456 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df0c134-fcb4-478e-a3de-44fb282dce9e None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:46.621212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-0df0c134-fcb4-478e-a3de-44fb282dce9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:46.621212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:46.621212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:46.621212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:46.621212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:46.621212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:46.621212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:46.621212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:46.621212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:46.621212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:46.621212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:46.621212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:46.621212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:46.621212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:46.621212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:46.621212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:46.623063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:46.623063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:46.623063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:46.623063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:46.623063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:46.623063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:46.623063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:46.623063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:46.623063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:46.623063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:46.623063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:46.623063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:46.623063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:46.623063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:46.623063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:46.623063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:46.623063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:46.624000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:46.624000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:46.624000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:46.624000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 279/562] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:46.624230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f76d2-0fa6-4df3-8b66-0b0d8b12041a None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:46.624308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f76d2-0fa6-4df3-8b66-0b0d8b12041a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:46.624583 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f76d2-0fa6-4df3-8b66-0b0d8b12041a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:46.644625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a80f76d2-0fa6-4df3-8b66-0b0d8b12041a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:46.645596 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a80f76d2-0fa6-4df3-8b66-0b0d8b12041a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:46.691915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:46.698781 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff03d5e-bd76-439f-a4f9-fa1a5a3b995f None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:46.731414 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 280/563] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:46.732472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:46.732788 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f76d2-0fa6-4df3-8b66-0b0d8b12041a None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:46.746472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2892afc-7e13-45ac-b1a4-c28d552c0146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:46.747121 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2892afc-7e13-45ac-b1a4-c28d552c0146 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:46.747401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2892afc-7e13-45ac-b1a4-c28d552c0146 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:46.747691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2892afc-7e13-45ac-b1a4-c28d552c0146 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:46.748311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2892afc-7e13-45ac-b1a4-c28d552c0146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:46.748607 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2892afc-7e13-45ac-b1a4-c28d552c0146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:46.750012 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2892afc-7e13-45ac-b1a4-c28d552c0146 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:46.762546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 284/564] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:46.763381 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2892afc-7e13-45ac-b1a4-c28d552c0146 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:46.842744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f76d2-0fa6-4df3-8b66-0b0d8b12041a None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:46.874666 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2892afc-7e13-45ac-b1a4-c28d552c0146 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:46.877952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed63b02-9bee-46f7-8146-4dd9366aa984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:46.879403 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed63b02-9bee-46f7-8146-4dd9366aa984 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:46.879901 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed63b02-9bee-46f7-8146-4dd9366aa984 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:46.880195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed63b02-9bee-46f7-8146-4dd9366aa984 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:46.880852 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed63b02-9bee-46f7-8146-4dd9366aa984 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:46.881154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed63b02-9bee-46f7-8146-4dd9366aa984 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:46.882544 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed63b02-9bee-46f7-8146-4dd9366aa984 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:46.899618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 281/565] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:46.901039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077157d1-3b45-42fb-9570-8476d21a0cc3 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:46.901311 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077157d1-3b45-42fb-9570-8476d21a0cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:46.901585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077157d1-3b45-42fb-9570-8476d21a0cc3 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:46.902907 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 282/566] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:46.908472 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077157d1-3b45-42fb-9570-8476d21a0cc3 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:46.921703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b1ed4-9bb4-4fc0-8bc0-6d6d8f24ae05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:46.922435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b1ed4-9bb4-4fc0-8bc0-6d6d8f24ae05 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:46.922766 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b1ed4-9bb4-4fc0-8bc0-6d6d8f24ae05 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:46.923072 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b1ed4-9bb4-4fc0-8bc0-6d6d8f24ae05 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:46.923772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b1ed4-9bb4-4fc0-8bc0-6d6d8f24ae05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:46.924072 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b1ed4-9bb4-4fc0-8bc0-6d6d8f24ae05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:46.926160 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b1ed4-9bb4-4fc0-8bc0-6d6d8f24ae05 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:46.945215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 285/567] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:46.946177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b1ed4-9bb4-4fc0-8bc0-6d6d8f24ae05 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:46.948953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:47.003546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:47.033517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077157d1-3b45-42fb-9570-8476d21a0cc3 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:47.062372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b1ed4-9bb4-4fc0-8bc0-6d6d8f24ae05 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:47.069889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71daaa8f-7ade-4fa2-9403-c5dcdf01150d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:47.071343 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daaa8f-7ade-4fa2-9403-c5dcdf01150d None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:47.071343 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daaa8f-7ade-4fa2-9403-c5dcdf01150d None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:47.071343 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daaa8f-7ade-4fa2-9403-c5dcdf01150d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:47.079146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71daaa8f-7ade-4fa2-9403-c5dcdf01150d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:47.080628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71daaa8f-7ade-4fa2-9403-c5dcdf01150d None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:47.093261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-71daaa8f-7ade-4fa2-9403-c5dcdf01150d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 19:05:47.093261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:47.093261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:47.093261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:47.093261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:47.093261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:47.093261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:47.093261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:47.093261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:47.093261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:47.093261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:47.093261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:47.093261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:47.093261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:47.093261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:47.093261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:47.095211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:47.097987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:47.097987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:47.097987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:47.097987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:47.097987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:47.097987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:47.097987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:47.097987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:47.097987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 19:05:47.097987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:47.097987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 283/568] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:47.097987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b4a3-17e5-4d3c-ae23-a941a1a33898 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:47.097987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b4a3-17e5-4d3c-ae23-a941a1a33898 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:47.100138 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b4a3-17e5-4d3c-ae23-a941a1a33898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:47.107883 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b8517-4d91-41ec-bb46-b0b2331b5d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:47.109264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b8517-4d91-41ec-bb46-b0b2331b5d9b None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:47.109264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b8517-4d91-41ec-bb46-b0b2331b5d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:47.109264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b8517-4d91-41ec-bb46-b0b2331b5d9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:47.116906 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b8517-4d91-41ec-bb46-b0b2331b5d9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:47.118139 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-85d7b4a3-17e5-4d3c-ae23-a941a1a33898 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:47.118354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b8517-4d91-41ec-bb46-b0b2331b5d9b None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:47.119527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-85d7b4a3-17e5-4d3c-ae23-a941a1a33898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:47.131711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-141b8517-4d91-41ec-bb46-b0b2331b5d9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:47.131711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:47.131711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:47.131711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:47.131711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:47.131711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:47.131711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:47.131711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:47.131711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:47.131711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:47.131711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:47.131711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:47.131711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:47.131711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:47.131711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:47.132912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:47.132912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:47.132912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:47.132912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:47.132912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:47.132912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:47.132912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:47.132912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:47.132912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:47.132912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:47.132912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:47.132912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:47.132912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:47.132912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:47.132912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:47.132912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:47.132912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:47.134826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:47.134826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:47.134826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:47.134826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:47.134826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:47.134826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:47.134826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:47.134826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:47.134826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 286/569] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:47.134826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b8517-4d91-41ec-bb46-b0b2331b5d9b None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:47.202153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 284/570] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:47.202840 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7b4a3-17e5-4d3c-ae23-a941a1a33898 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:47.234996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:47.262860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b8517-4d91-41ec-bb46-b0b2331b5d9b None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:47.266759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:47.305635 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ed1a2-837f-47a6-ab88-f1c396d35b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:47.306273 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ed1a2-837f-47a6-ab88-f1c396d35b19 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:47.306555 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ed1a2-837f-47a6-ab88-f1c396d35b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:47.306855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ed1a2-837f-47a6-ab88-f1c396d35b19 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:47.315887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ed1a2-837f-47a6-ab88-f1c396d35b19 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:47.317954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ed1a2-837f-47a6-ab88-f1c396d35b19 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:47.332079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-e06ed1a2-837f-47a6-ab88-f1c396d35b19 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:05:47.332079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:47.332079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:47.332079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:47.332079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:47.332079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:47.332079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:47.332079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:47.332079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:47.332079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:47.332079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:47.332079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:47.332079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:47.332079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:47.332079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:47.332079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:47.333146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:47.334116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:47.334116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:47.334116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:47.334116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:47.334116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:47.334116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:47.334116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:47.334116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:47.334116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:05:47.334116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:47.334116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 287/571] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 31 19:05:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:47.334709 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ed1a2-837f-47a6-ab88-f1c396d35b19 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:47.372118 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7b4a3-17e5-4d3c-ae23-a941a1a33898 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:47.412769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de7fec-cb98-4eca-a3ac-6e4e2db08d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:47.413846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de7fec-cb98-4eca-a3ac-6e4e2db08d6b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:47.414304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de7fec-cb98-4eca-a3ac-6e4e2db08d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:47.414750 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de7fec-cb98-4eca-a3ac-6e4e2db08d6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:47.426365 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de7fec-cb98-4eca-a3ac-6e4e2db08d6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:47.431532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-f3de7fec-cb98-4eca-a3ac-6e4e2db08d6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:47.431532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:47.431532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:47.431532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:47.431532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:47.431532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:47.431532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:47.431532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:47.431532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:47.431532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:47.431532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:47.431532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:47.431532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:47.431532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:47.431532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:47.431532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:47.432466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:47.432466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:47.432466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:47.432466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:47.432466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:47.432466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:47.432466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:47.432466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:47.432466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:47.432466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:47.432466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:47.432466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:47.432466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:47.432466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:47.432466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:47.432466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:47.433318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:47.433318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:47.433318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:47.433318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:47.433318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:47.433318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:47.433318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:47.433318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:47.433318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:47.433318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:47.433318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:47.433318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:47.439712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 285/572] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:47.440160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de7fec-cb98-4eca-a3ac-6e4e2db08d6b None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:47.452902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ed1a2-837f-47a6-ab88-f1c396d35b19 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:47.495878 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068fc11-ff1b-44a7-a069-88553fd669ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:47.495878 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068fc11-ff1b-44a7-a069-88553fd669ec None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:47.495878 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068fc11-ff1b-44a7-a069-88553fd669ec None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:47.496869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068fc11-ff1b-44a7-a069-88553fd669ec None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:47.496869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7068fc11-ff1b-44a7-a069-88553fd669ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:47.497170 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7068fc11-ff1b-44a7-a069-88553fd669ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:47.498627 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7068fc11-ff1b-44a7-a069-88553fd669ec None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:47.516909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 288/573] 10.222.0.59 () {56 vars in 1137 bytes} [Wed Aug 31 19:05:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:47.517732 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068fc11-ff1b-44a7-a069-88553fd669ec None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:47.533688 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:47.542568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:47.548780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de7fec-cb98-4eca-a3ac-6e4e2db08d6b None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:47.586591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f7309-cc9c-47c0-a644-f8661e64fd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:47.587256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f7309-cc9c-47c0-a644-f8661e64fd8e None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:47.587534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f7309-cc9c-47c0-a644-f8661e64fd8e None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:47.587808 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f7309-cc9c-47c0-a644-f8661e64fd8e None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:47.588496 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f7309-cc9c-47c0-a644-f8661e64fd8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:47.588798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f7309-cc9c-47c0-a644-f8661e64fd8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:47.591068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f7309-cc9c-47c0-a644-f8661e64fd8e None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:47.602485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 286/574] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:47.603318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f7309-cc9c-47c0-a644-f8661e64fd8e None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:47.632166 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068fc11-ff1b-44a7-a069-88553fd669ec None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:47.666316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c9d80-cfc8-4c81-bee6-1755190fdb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:47.666951 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c9d80-cfc8-4c81-bee6-1755190fdb38 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:47.667237 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c9d80-cfc8-4c81-bee6-1755190fdb38 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:47.667507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c9d80-cfc8-4c81-bee6-1755190fdb38 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:47.668239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c9d80-cfc8-4c81-bee6-1755190fdb38 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:47.668547 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c9d80-cfc8-4c81-bee6-1755190fdb38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:47.669971 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c9d80-cfc8-4c81-bee6-1755190fdb38 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:47.687167 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 289/575] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 31 19:05:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:47.688153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c9d80-cfc8-4c81-bee6-1755190fdb38 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:47.701974 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f7309-cc9c-47c0-a644-f8661e64fd8e None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:47.739311 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81a300-630d-4734-9047-f16fd1b3d513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:47.739927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81a300-630d-4734-9047-f16fd1b3d513 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:47.740205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81a300-630d-4734-9047-f16fd1b3d513 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:47.740475 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81a300-630d-4734-9047-f16fd1b3d513 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:47.741139 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81a300-630d-4734-9047-f16fd1b3d513 None admin] RBAC: Adding query filter params (name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:47.741438 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81a300-630d-4734-9047-f16fd1b3d513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:47.742807 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81a300-630d-4734-9047-f16fd1b3d513 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:47.751036 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 287/576] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:05:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:47.751855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81a300-630d-4734-9047-f16fd1b3d513 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:47.765240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:47.781689 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:47.801961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c9d80-cfc8-4c81-bee6-1755190fdb38 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:47.836313 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d04090-9b6d-4a05-9b62-9b385b70e23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:47.836954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d04090-9b6d-4a05-9b62-9b385b70e23d None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:47.837231 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d04090-9b6d-4a05-9b62-9b385b70e23d None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:47.837506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d04090-9b6d-4a05-9b62-9b385b70e23d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:47.851916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d04090-9b6d-4a05-9b62-9b385b70e23d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:47.856828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-15d04090-9b6d-4a05-9b62-9b385b70e23d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:47.856828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:47.856828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:47.856828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:47.856828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:47.856828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:47.856828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:47.856828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:47.856828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:47.856828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:47.856828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:47.856828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:47.856828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:47.856828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:47.856828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:47.856828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:47.857780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:47.857780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:47.857780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:47.857780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:47.857780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:47.857780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:47.857780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:47.857780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:47.857780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:47.857780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:47.857780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:47.857780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:47.857780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:47.857780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:47.857780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:47.857780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:47.858649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:47.858649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:47.858649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:47.858649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:47.858649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:47.858649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:47.858649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:47.858649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:47.858649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:47.858649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:47.858649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:47.858649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:47.858649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 290/577] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:47.859389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d04090-9b6d-4a05-9b62-9b385b70e23d None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:47.862708 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81a300-630d-4734-9047-f16fd1b3d513 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:47.898583 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd9a89-36a9-49a6-a863-9eaaede474b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:47.899172 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd9a89-36a9-49a6-a863-9eaaede474b9 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:47.899456 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd9a89-36a9-49a6-a863-9eaaede474b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:47.899732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd9a89-36a9-49a6-a863-9eaaede474b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:47.912851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd9a89-36a9-49a6-a863-9eaaede474b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:47.914320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd9a89-36a9-49a6-a863-9eaaede474b9 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:47.929738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-d9fd9a89-36a9-49a6-a863-9eaaede474b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:47.929738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:47.929738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:47.929738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:47.929738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:47.929738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:47.929738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:47.929738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:47.929738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:47.929738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:47.929738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:47.929738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:47.929738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:47.929738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:47.929738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:47.929738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:47.930810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:47.930810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:47.930810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:47.930810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:47.930810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:47.930810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:47.930810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:47.930810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:47.930810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:47.930810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:47.930810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:47.930810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:47.930810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:47.930810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:47.930810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:47.930810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:47.930810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:47.931793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:47.931793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:47.931793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:47.931793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 288/578] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:47.932255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd9a89-36a9-49a6-a863-9eaaede474b9 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:47.971355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d04090-9b6d-4a05-9b62-9b385b70e23d None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:47.992931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:48.005538 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15527df7-c5b3-4924-9731-279310890ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:48.006149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15527df7-c5b3-4924-9731-279310890ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:48.006428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15527df7-c5b3-4924-9731-279310890ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:48.006701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15527df7-c5b3-4924-9731-279310890ab6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:48.007458 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15527df7-c5b3-4924-9731-279310890ab6 None admin] RBAC: Adding query filter params (role.id=a9ef0098e982437fb0127827fa2cad38, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=25ed58369ffd46e8b7fa1880662926e8) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:48.007768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15527df7-c5b3-4924-9731-279310890ab6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:48.009195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15527df7-c5b3-4924-9731-279310890ab6 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:48.014545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 291/579] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:47 2022] GET /identity/v3/role_assignments?user.id=25ed58369ffd46e8b7fa1880662926e8&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=a9ef0098e982437fb0127827fa2cad38 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:48.015373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15527df7-c5b3-4924-9731-279310890ab6 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:48.020160 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:48.048527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd9a89-36a9-49a6-a863-9eaaede474b9 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:48.090368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8edc0-9c4d-4e20-8ced-9389b9be9453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:48.091224 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8edc0-9c4d-4e20-8ced-9389b9be9453 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:48.091512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8edc0-9c4d-4e20-8ced-9389b9be9453 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:48.091793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8edc0-9c4d-4e20-8ced-9389b9be9453 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:48.104676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8edc0-9c4d-4e20-8ced-9389b9be9453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:48.106169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8edc0-9c4d-4e20-8ced-9389b9be9453 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:48.123861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-e7b8edc0-9c4d-4e20-8ced-9389b9be9453 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:48.123861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:48.123861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:48.123861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:48.123861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:48.123861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:48.123861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:48.123861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:48.123861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:48.123861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:48.123861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:48.123861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:48.123861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:48.123861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:48.123861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:48.123861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:48.124909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:48.124909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:48.124909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:48.124909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:48.124909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:48.124909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:48.124909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:48.124909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:48.124909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:48.124909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:48.124909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:48.124909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:48.124909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:48.124909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:48.124909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:48.124909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:48.124909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:48.125917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:48.125917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:48.125917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:48.125917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 289/580] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:48.126157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8edc0-9c4d-4e20-8ced-9389b9be9453 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:48.145074 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15527df7-c5b3-4924-9731-279310890ab6 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:48.186203 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdbb5a-e134-45a3-9bc0-e391442de41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:48.186797 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdbb5a-e134-45a3-9bc0-e391442de41f None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:48.187108 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdbb5a-e134-45a3-9bc0-e391442de41f None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:48.187385 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdbb5a-e134-45a3-9bc0-e391442de41f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:48.199303 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcdbb5a-e134-45a3-9bc0-e391442de41f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:48.200741 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcdbb5a-e134-45a3-9bc0-e391442de41f None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:48.216280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-1fcdbb5a-e134-45a3-9bc0-e391442de41f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:48.216280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:48.216280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:48.216280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:48.216280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:48.216280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:48.216280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:48.216280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:48.216280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:48.216280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:48.216280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:48.216280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:48.216280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:48.216280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:48.216280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:48.216280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:48.217353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:48.217353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:48.217353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:48.217353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:48.217353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:48.217353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:48.217353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:48.217353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:48.217353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:48.217353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:48.217353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:48.217353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:48.217353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:48.217353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:48.217353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:48.217353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:48.217353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:48.218314 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:48.218314 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:48.218314 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:48.218314 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 292/581] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:48.219872 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdbb5a-e134-45a3-9bc0-e391442de41f None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:48.257724 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8edc0-9c4d-4e20-8ced-9389b9be9453 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:48.275728 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:48.288562 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:48.300574 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcb772-cc60-4118-bef0-bc89f910baf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:48.301368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcb772-cc60-4118-bef0-bc89f910baf3 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:48.301631 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcb772-cc60-4118-bef0-bc89f910baf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:48.301909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcb772-cc60-4118-bef0-bc89f910baf3 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:48.302596 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcb772-cc60-4118-bef0-bc89f910baf3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:48.302944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcb772-cc60-4118-bef0-bc89f910baf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:48.304362 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcb772-cc60-4118-bef0-bc89f910baf3 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:48.314138 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 290/582] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:05:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:48.315099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcb772-cc60-4118-bef0-bc89f910baf3 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:48.389356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdbb5a-e134-45a3-9bc0-e391442de41f None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:48.425453 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f0d8b-511d-4984-a8ef-9c4e0e6abc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:48.426076 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f0d8b-511d-4984-a8ef-9c4e0e6abc2c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:48.426356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f0d8b-511d-4984-a8ef-9c4e0e6abc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:48.426632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f0d8b-511d-4984-a8ef-9c4e0e6abc2c None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:48.427309 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f0d8b-511d-4984-a8ef-9c4e0e6abc2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:48.427620 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f0d8b-511d-4984-a8ef-9c4e0e6abc2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:48.428986 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f0d8b-511d-4984-a8ef-9c4e0e6abc2c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:48.446542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 293/583] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:48.447370 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f0d8b-511d-4984-a8ef-9c4e0e6abc2c None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:48.472643 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcb772-cc60-4118-bef0-bc89f910baf3 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:48.515837 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82c1ce-f84d-40a3-bf91-f84655ef8781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:48.516460 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82c1ce-f84d-40a3-bf91-f84655ef8781 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:48.516917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82c1ce-f84d-40a3-bf91-f84655ef8781 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:48.517206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82c1ce-f84d-40a3-bf91-f84655ef8781 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:48.517856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82c1ce-f84d-40a3-bf91-f84655ef8781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:48.518159 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82c1ce-f84d-40a3-bf91-f84655ef8781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:48.520107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82c1ce-f84d-40a3-bf91-f84655ef8781 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:48.537610 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 291/584] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:48.539245 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82c1ce-f84d-40a3-bf91-f84655ef8781 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:48.562440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f0d8b-511d-4984-a8ef-9c4e0e6abc2c None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:48.575610 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:48.582729 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:48.600994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6fa0b-9b42-479b-a20a-547f6fd6868f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:48.601612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6fa0b-9b42-479b-a20a-547f6fd6868f None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:48.601888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6fa0b-9b42-479b-a20a-547f6fd6868f None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:48.602211 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6fa0b-9b42-479b-a20a-547f6fd6868f None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:48.602850 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6fa0b-9b42-479b-a20a-547f6fd6868f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:48.603355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6fa0b-9b42-479b-a20a-547f6fd6868f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:48.604754 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6fa0b-9b42-479b-a20a-547f6fd6868f None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:48.616107 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 294/585] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:48.617528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6fa0b-9b42-479b-a20a-547f6fd6868f None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:48.656485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82c1ce-f84d-40a3-bf91-f84655ef8781 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:48.696384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af268fe-9837-456e-b547-ede60f47f0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:48.697077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af268fe-9837-456e-b547-ede60f47f0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:48.697308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af268fe-9837-456e-b547-ede60f47f0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:48.697583 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af268fe-9837-456e-b547-ede60f47f0e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:48.709205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af268fe-9837-456e-b547-ede60f47f0e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:48.710651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af268fe-9837-456e-b547-ede60f47f0e1 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:48.722688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6fa0b-9b42-479b-a20a-547f6fd6868f None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:48.726171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-2af268fe-9837-456e-b547-ede60f47f0e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:48.726171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:48.726171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:48.726171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:48.726171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:48.726171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:48.726171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:48.726171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:48.726171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:48.726171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:48.726171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:48.726171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:48.726171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:48.726171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:48.726171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:48.726171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:48.732194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:48.732194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:48.732194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:48.732194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:48.732194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:48.732194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:48.732194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:48.732194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:48.732194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:48.732194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:48.732194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:48.732194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:48.732194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:48.732194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:48.732194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:48.732194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:48.732194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:48.733180 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:48.733180 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:48.733180 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:48.733180 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 292/586] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:48.733180 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af268fe-9837-456e-b547-ede60f47f0e1 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:48.762841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b751bb-62d6-479a-8c78-2108220c312a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:48.763509 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b751bb-62d6-479a-8c78-2108220c312a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:48.763789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b751bb-62d6-479a-8c78-2108220c312a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:48.764066 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b751bb-62d6-479a-8c78-2108220c312a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:48.771572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b751bb-62d6-479a-8c78-2108220c312a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:48.773002 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b751bb-62d6-479a-8c78-2108220c312a None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:48.784240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-83b751bb-62d6-479a-8c78-2108220c312a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:48.784240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:48.784240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:48.784240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:48.784240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:48.784240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:48.784240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:48.784240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:48.784240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:48.784240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:48.784240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:48.784240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:48.784240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:48.784240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:48.784240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:48.785798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:48.785798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:48.785798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:48.785798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:48.785798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:48.785798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:48.785798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:48.785798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:48.785798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:48.785798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:48.785798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:48.785798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:48.785798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:48.785798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:48.785798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:48.785798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:48.785798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:48.786759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:48.786759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:48.786759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:48.786759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:48.786759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:48.786759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:48.786759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:48.786759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:48.790984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 295/587] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:48.791826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b751bb-62d6-479a-8c78-2108220c312a None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:48.823328 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:48.834361 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:48.838448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af268fe-9837-456e-b547-ede60f47f0e1 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:48.877256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7835c-a9ae-4323-8686-9e97d897049a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:48.877893 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7835c-a9ae-4323-8686-9e97d897049a None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:48.878262 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7835c-a9ae-4323-8686-9e97d897049a None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:48.879052 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7835c-a9ae-4323-8686-9e97d897049a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:48.887452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7835c-a9ae-4323-8686-9e97d897049a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:48.888894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7835c-a9ae-4323-8686-9e97d897049a None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:48.901085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-89b7835c-a9ae-4323-8686-9e97d897049a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:48.901085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:48.901085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:48.901085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:48.901085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:48.901085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:48.901085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:48.901085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:48.901085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:48.901085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:48.901085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:48.901085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:48.901085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:48.901085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:48.901085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:48.902227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:48.902227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:48.902227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:48.902227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:48.902227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:48.902227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:48.902227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:48.902227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:48.902227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:48.902227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:48.902227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:48.902227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:48.902227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:48.902227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:48.902227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:48.902227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:48.902227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:48.903741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:48.903741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:48.903741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:48.903741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:48.903741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:48.903741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:48.903741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:48.903741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:48.906846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 293/588] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:48.907688 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7835c-a9ae-4323-8686-9e97d897049a None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:48.908157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b751bb-62d6-479a-8c78-2108220c312a None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:48.947487 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00439f75-7053-49f7-bab4-5df6afdec3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:48.948026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00439f75-7053-49f7-bab4-5df6afdec3a3 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:48.948302 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00439f75-7053-49f7-bab4-5df6afdec3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:48.948568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00439f75-7053-49f7-bab4-5df6afdec3a3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:48.957387 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00439f75-7053-49f7-bab4-5df6afdec3a3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:48.958861 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00439f75-7053-49f7-bab4-5df6afdec3a3 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:48.970182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-00439f75-7053-49f7-bab4-5df6afdec3a3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:05:48.970182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:48.970182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:48.970182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:48.970182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:48.970182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:48.970182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:48.970182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:48.970182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:48.970182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:48.970182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:48.970182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:48.970182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:48.970182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:48.970182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:48.970182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:48.971202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:48.973688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:48.973688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:48.973688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:48.973688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:48.973688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:48.973688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:48.973688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:48.973688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:48.973688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:05:48.973688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:48.973688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 296/589] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:48.973688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00439f75-7053-49f7-bab4-5df6afdec3a3 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:49.023017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7835c-a9ae-4323-8686-9e97d897049a None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:49.056452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f37894-dc69-4dbd-8e90-162cbd2cc4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:49.057060 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f37894-dc69-4dbd-8e90-162cbd2cc4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:49.057338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f37894-dc69-4dbd-8e90-162cbd2cc4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:49.057614 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f37894-dc69-4dbd-8e90-162cbd2cc4a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:49.058255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f37894-dc69-4dbd-8e90-162cbd2cc4a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:49.058556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f37894-dc69-4dbd-8e90-162cbd2cc4a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:49.059978 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f37894-dc69-4dbd-8e90-162cbd2cc4a9 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:49.076744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 294/590] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:49.076744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f37894-dc69-4dbd-8e90-162cbd2cc4a9 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:49.082235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:49.090654 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:49.097727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00439f75-7053-49f7-bab4-5df6afdec3a3 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:49.138707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb514a8-d570-4a67-ac37-c81a76645f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:49.139361 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb514a8-d570-4a67-ac37-c81a76645f9a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:49.139641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb514a8-d570-4a67-ac37-c81a76645f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:49.139915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb514a8-d570-4a67-ac37-c81a76645f9a None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:49.140601 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb514a8-d570-4a67-ac37-c81a76645f9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:49.140908 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb514a8-d570-4a67-ac37-c81a76645f9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:49.142332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb514a8-d570-4a67-ac37-c81a76645f9a None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:49.156176 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 297/591] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:49.156906 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb514a8-d570-4a67-ac37-c81a76645f9a None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:49.188956 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f37894-dc69-4dbd-8e90-162cbd2cc4a9 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:49.224829 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb534bf3-6d7f-4d9c-862a-492eff4dcf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:49.225281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb534bf3-6d7f-4d9c-862a-492eff4dcf72 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:49.225538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb534bf3-6d7f-4d9c-862a-492eff4dcf72 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:49.225781 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb534bf3-6d7f-4d9c-862a-492eff4dcf72 None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:49.226479 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb534bf3-6d7f-4d9c-862a-492eff4dcf72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:49.226790 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb534bf3-6d7f-4d9c-862a-492eff4dcf72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:49.228254 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb534bf3-6d7f-4d9c-862a-492eff4dcf72 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:49.245968 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 295/592] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:49.245968 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb534bf3-6d7f-4d9c-862a-492eff4dcf72 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:49.268382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb514a8-d570-4a67-ac37-c81a76645f9a None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:49.307412 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9a982-bbe0-49b2-8d91-8400e862285d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:49.308019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9a982-bbe0-49b2-8d91-8400e862285d None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:49.308294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9a982-bbe0-49b2-8d91-8400e862285d None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:49.308563 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9a982-bbe0-49b2-8d91-8400e862285d None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:49.309294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9a982-bbe0-49b2-8d91-8400e862285d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:49.309592 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9a982-bbe0-49b2-8d91-8400e862285d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:49.311015 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9a982-bbe0-49b2-8d91-8400e862285d None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:49.321324 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:49.330118 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 298/593] 10.222.0.59 () {56 vars in 1137 bytes} [Wed Aug 31 19:05:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:49.331054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9a982-bbe0-49b2-8d91-8400e862285d None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:49.347843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:49.360638 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb534bf3-6d7f-4d9c-862a-492eff4dcf72 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:49.400639 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78d9ee-c543-4879-b982-6afaa015bf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:49.401446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78d9ee-c543-4879-b982-6afaa015bf1c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:49.401831 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78d9ee-c543-4879-b982-6afaa015bf1c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:49.402209 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78d9ee-c543-4879-b982-6afaa015bf1c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:49.411190 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78d9ee-c543-4879-b982-6afaa015bf1c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:49.412769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78d9ee-c543-4879-b982-6afaa015bf1c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:49.426361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-4c78d9ee-c543-4879-b982-6afaa015bf1c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 19:05:49.426361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:49.426361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:49.426361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:49.426361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:49.426361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:49.426361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:49.426361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:49.426361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:49.426361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:49.426361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:49.426361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:49.426361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:49.426361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:49.426361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:49.426361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:49.427578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:49.428538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:49.428538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:49.428538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:49.428538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:49.428538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:49.428538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:49.428538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:49.428538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:49.428538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 19:05:49.428538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:49.428538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 296/594] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:49.428538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78d9ee-c543-4879-b982-6afaa015bf1c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:49.447844 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9a982-bbe0-49b2-8d91-8400e862285d None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:49.490649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0815a31-6334-440b-9eee-238ca30e8b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:49.491336 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0815a31-6334-440b-9eee-238ca30e8b03 None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:49.491617 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0815a31-6334-440b-9eee-238ca30e8b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:49.491883 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0815a31-6334-440b-9eee-238ca30e8b03 None admin] PATH_INFO: `/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/f281d778f1404984b4d778a1c26aef1d/roles/a9ef0098e982437fb0127827fa2cad38` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:49.507455 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0815a31-6334-440b-9eee-238ca30e8b03 None admin] RBAC: Authorizing `identity:create_grant(project_id=161c90d34bb34925a1aaeb70c7de7eae, user_id=f281d778f1404984b4d778a1c26aef1d, role_id=a9ef0098e982437fb0127827fa2cad38)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:49.508926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0815a31-6334-440b-9eee-238ca30e8b03 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:49.534355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78d9ee-c543-4879-b982-6afaa015bf1c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:49.559996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:49.568354 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc8356-b42f-4e1f-b330-c366cdc2fb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:49.569013 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc8356-b42f-4e1f-b330-c366cdc2fb7c None admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:49.569292 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc8356-b42f-4e1f-b330-c366cdc2fb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:49.569760 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc8356-b42f-4e1f-b330-c366cdc2fb7c None admin] PATH_INFO: `/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/d461422ce98442bcbe6b254a8dad5e8a/roles/a9ef0098e982437fb0127827fa2cad38` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:49.574497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 299/595] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:05:49 2022] PUT /identity/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/f281d778f1404984b4d778a1c26aef1d/roles/a9ef0098e982437fb0127827fa2cad38 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:05:49.579938 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0815a31-6334-440b-9eee-238ca30e8b03 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:49.586796 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc8356-b42f-4e1f-b330-c366cdc2fb7c None admin] RBAC: Authorizing `identity:create_grant(project_id=161c90d34bb34925a1aaeb70c7de7eae, user_id=d461422ce98442bcbe6b254a8dad5e8a, role_id=a9ef0098e982437fb0127827fa2cad38)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:49.588366 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc8356-b42f-4e1f-b330-c366cdc2fb7c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:49.667039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:49.676491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 297/596] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:05:49 2022] PUT /identity/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/d461422ce98442bcbe6b254a8dad5e8a/roles/a9ef0098e982437fb0127827fa2cad38 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:05:49.678493 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc8356-b42f-4e1f-b330-c366cdc2fb7c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:49.748440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0815a31-6334-440b-9eee-238ca30e8b03 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:49.798204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c87542-9136-4ee9-8a5a-5c55c2d7a886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:49.799577 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c87542-9136-4ee9-8a5a-5c55c2d7a886 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:49.799864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c87542-9136-4ee9-8a5a-5c55c2d7a886 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:49.800876 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c87542-9136-4ee9-8a5a-5c55c2d7a886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:49.814002 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c87542-9136-4ee9-8a5a-5c55c2d7a886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:49.815691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c87542-9136-4ee9-8a5a-5c55c2d7a886 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:49.835711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-19c87542-9136-4ee9-8a5a-5c55c2d7a886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:49.835711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:49.835711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:49.835711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:49.835711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:49.835711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:49.835711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:49.835711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:49.835711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:49.835711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:49.835711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:49.835711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:49.835711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:49.835711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:49.835711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:49.835711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:49.836746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:49.836746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:49.836746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:49.836746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:49.836746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:49.836746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:49.836746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:49.836746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:49.836746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:49.836746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:49.836746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:49.836746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:49.836746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:49.836746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:49.836746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:49.836746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:49.836746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:49.837678 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:49.837678 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:49.837678 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:49.837855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc8356-b42f-4e1f-b330-c366cdc2fb7c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:49.837942 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 300/597] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:49.848751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c87542-9136-4ee9-8a5a-5c55c2d7a886 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:49.878296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1afba-3905-43c3-b61a-23e14a87acdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:49.882828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1afba-3905-43c3-b61a-23e14a87acdd None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:49.882828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1afba-3905-43c3-b61a-23e14a87acdd None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:49.882828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1afba-3905-43c3-b61a-23e14a87acdd None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:49.882828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1afba-3905-43c3-b61a-23e14a87acdd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:49.882828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1afba-3905-43c3-b61a-23e14a87acdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:49.882828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1afba-3905-43c3-b61a-23e14a87acdd None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:49.904555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 298/598] 10.222.0.59 () {56 vars in 1139 bytes} [Wed Aug 31 19:05:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:49.910828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1afba-3905-43c3-b61a-23e14a87acdd None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:49.950707 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:49.996033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c87542-9136-4ee9-8a5a-5c55c2d7a886 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:50.014662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:50.041959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc1109-7e90-4c08-b438-9ace14b2ffca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:50.042891 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc1109-7e90-4c08-b438-9ace14b2ffca None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:50.043368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc1109-7e90-4c08-b438-9ace14b2ffca None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:50.043656 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc1109-7e90-4c08-b438-9ace14b2ffca None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:50.044330 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc1109-7e90-4c08-b438-9ace14b2ffca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:50.044635 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc1109-7e90-4c08-b438-9ace14b2ffca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:50.046163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc1109-7e90-4c08-b438-9ace14b2ffca None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:50.059681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1afba-3905-43c3-b61a-23e14a87acdd None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:50.060520 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 301/599] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:50.062377 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e9291-3e1d-4626-876c-a23c7c6c3c7f None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:50.062762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e9291-3e1d-4626-876c-a23c7c6c3c7f None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:50.063158 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e9291-3e1d-4626-876c-a23c7c6c3c7f None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:50.064793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 302/600] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:50.067168 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e9291-3e1d-4626-876c-a23c7c6c3c7f None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:50.102964 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabea86-e71c-48c0-ac65-94195484baa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:50.103616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabea86-e71c-48c0-ac65-94195484baa7 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:50.103901 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabea86-e71c-48c0-ac65-94195484baa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:50.104177 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabea86-e71c-48c0-ac65-94195484baa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:50.116249 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddabea86-e71c-48c0-ac65-94195484baa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:50.117724 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddabea86-e71c-48c0-ac65-94195484baa7 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:50.133467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-ddabea86-e71c-48c0-ac65-94195484baa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:50.133467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:50.133467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:50.133467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:50.133467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:50.133467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:50.133467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:50.133467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:50.133467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:50.133467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:50.133467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:50.133467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:50.133467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:50.133467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:50.133467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:50.133467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:50.134487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:50.134487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:50.134487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:50.134487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:50.134487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:50.134487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:50.134487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:50.134487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:50.134487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:50.134487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:50.134487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:50.134487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:50.134487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:50.134487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:50.134487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:50.134487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:50.134487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:50.136032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:50.136032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:50.136032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:50.136032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 299/601] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:50.136290 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e12d9d-fd0c-4a37-80d2-c1ad6d0b403c None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:50.136469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e12d9d-fd0c-4a37-80d2-c1ad6d0b403c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:50.136698 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e12d9d-fd0c-4a37-80d2-c1ad6d0b403c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:50.156533 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-01e12d9d-fd0c-4a37-80d2-c1ad6d0b403c None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:50.157766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-01e12d9d-fd0c-4a37-80d2-c1ad6d0b403c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:50.188985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e9291-3e1d-4626-876c-a23c7c6c3c7f None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:50.226496 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d7842-33f4-4f7b-9fbe-f50a95b93e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:50.227198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d7842-33f4-4f7b-9fbe-f50a95b93e9e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:50.227475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d7842-33f4-4f7b-9fbe-f50a95b93e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:50.227748 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d7842-33f4-4f7b-9fbe-f50a95b93e9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:50.235433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d7842-33f4-4f7b-9fbe-f50a95b93e9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:50.237208 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d7842-33f4-4f7b-9fbe-f50a95b93e9e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:50.239282 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 300/602] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:50.240329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e12d9d-fd0c-4a37-80d2-c1ad6d0b403c None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:50.249215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-5f4d7842-33f4-4f7b-9fbe-f50a95b93e9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:50.249215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:50.249215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:50.249215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:50.249215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:50.249215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:50.249215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:50.249215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:50.249215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:50.249215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:50.249215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:50.249215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:50.249215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:50.249215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:50.249215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:50.250133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:50.250133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:50.250133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:50.250133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:50.250133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:50.250133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:50.250133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:50.250133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:50.250133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:50.250133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:50.250133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:50.250133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:50.250133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:50.250133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:50.250133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:50.250133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:50.250133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:50.251465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:50.251465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:50.251465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:50.251465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:50.251465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:50.251465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:50.251465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:50.251465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:50.251465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 303/603] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:50.256697 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d7842-33f4-4f7b-9fbe-f50a95b93e9e None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:50.283331 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:50.295688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:50.372578 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d7842-33f4-4f7b-9fbe-f50a95b93e9e None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:50.449950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff74465-5a0d-43b0-9c60-86ce8116bcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:50.450901 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff74465-5a0d-43b0-9c60-86ce8116bcb4 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:50.451355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff74465-5a0d-43b0-9c60-86ce8116bcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:50.451805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff74465-5a0d-43b0-9c60-86ce8116bcb4 None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:50.457447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff74465-5a0d-43b0-9c60-86ce8116bcb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:50.457789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff74465-5a0d-43b0-9c60-86ce8116bcb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:50.460219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff74465-5a0d-43b0-9c60-86ce8116bcb4 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:50.463321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e12d9d-fd0c-4a37-80d2-c1ad6d0b403c None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:50.494589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 304/604] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:50.519541 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff74465-5a0d-43b0-9c60-86ce8116bcb4 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:50.539244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8c838-a3aa-440d-aebb-f57b41dc8c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:50.540101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8c838-a3aa-440d-aebb-f57b41dc8c84 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:50.540552 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8c838-a3aa-440d-aebb-f57b41dc8c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:50.541004 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8c838-a3aa-440d-aebb-f57b41dc8c84 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:50.542429 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8c838-a3aa-440d-aebb-f57b41dc8c84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:50.542962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8c838-a3aa-440d-aebb-f57b41dc8c84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:50.547917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8c838-a3aa-440d-aebb-f57b41dc8c84 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:50.566359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 301/605] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:50.571701 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8c838-a3aa-440d-aebb-f57b41dc8c84 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:50.662588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:50.663401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:50.669785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff74465-5a0d-43b0-9c60-86ce8116bcb4 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:50.686261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8c838-a3aa-440d-aebb-f57b41dc8c84 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:50.713087 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d7c78-ecbf-44d4-809f-548e8b5328b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:50.713765 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d7c78-ecbf-44d4-809f-548e8b5328b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:50.714045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d7c78-ecbf-44d4-809f-548e8b5328b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:50.714424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d7c78-ecbf-44d4-809f-548e8b5328b2 None admin] PATH_INFO: `/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/108a698aa3e6477b9ae896d68f22bd0c/roles/a9ef0098e982437fb0127827fa2cad38` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:50.723727 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a8e94-6693-49d7-bbe0-02536894c35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:50.724368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a8e94-6693-49d7-bbe0-02536894c35c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:50.724652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a8e94-6693-49d7-bbe0-02536894c35c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:50.724920 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a8e94-6693-49d7-bbe0-02536894c35c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:50.731914 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d7c78-ecbf-44d4-809f-548e8b5328b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=161c90d34bb34925a1aaeb70c7de7eae, user_id=108a698aa3e6477b9ae896d68f22bd0c, role_id=a9ef0098e982437fb0127827fa2cad38)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:50.732356 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a8e94-6693-49d7-bbe0-02536894c35c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:50.733780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d7c78-ecbf-44d4-809f-548e8b5328b2 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:50.733960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a8e94-6693-49d7-bbe0-02536894c35c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:50.746187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-602a8e94-6693-49d7-bbe0-02536894c35c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:50.746187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:50.746187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:50.746187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:50.746187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:50.746187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:50.746187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:50.746187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:50.746187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:50.746187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:50.746187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:50.746187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:50.746187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:50.746187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:50.746187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:50.747308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:50.747308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:50.747308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:50.747308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:50.747308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:50.747308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:50.747308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:50.747308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:50.747308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:50.747308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:50.747308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:50.747308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:50.747308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:50.747308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:50.747308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:50.747308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:50.747308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:50.748512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:50.748512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:50.748512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:50.748512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:50.748512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:50.748512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:50.748512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:50.748512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:50.751969 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 302/606] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:50.752688 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a8e94-6693-49d7-bbe0-02536894c35c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:50.829427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 305/607] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:05:50 2022] PUT /identity/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/108a698aa3e6477b9ae896d68f22bd0c/roles/a9ef0098e982437fb0127827fa2cad38 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:05:50.830745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d7c78-ecbf-44d4-809f-548e8b5328b2 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:50.883539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a8e94-6693-49d7-bbe0-02536894c35c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:50.954062 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c096ac-90a3-41f0-8fcf-d3005c4e7a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:50.954448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c096ac-90a3-41f0-8fcf-d3005c4e7a16 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:50.954740 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c096ac-90a3-41f0-8fcf-d3005c4e7a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:50.955051 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c096ac-90a3-41f0-8fcf-d3005c4e7a16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:50.959341 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:50.976290 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d7c78-ecbf-44d4-809f-548e8b5328b2 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:50.991239 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c096ac-90a3-41f0-8fcf-d3005c4e7a16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:51.004622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-22c096ac-90a3-41f0-8fcf-d3005c4e7a16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:51.004622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:51.004622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:51.004622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:51.004622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:51.004622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:51.004622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:51.004622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:51.004622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:51.004622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:51.004622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:51.004622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:51.004622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:51.004622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:51.004622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:51.004622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:51.005946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:51.005946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:51.005946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:51.005946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:51.005946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:51.005946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:51.005946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:51.005946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:51.005946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:51.005946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:51.005946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:51.005946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:51.005946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:51.005946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:51.005946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:51.005946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:51.011206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:51.011206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:51.011206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:51.011206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:51.011206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:51.011206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:51.011206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:51.011206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:51.011206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:51.011206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:51.011206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:51.011206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:51.011206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 303/608] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:51.021183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c096ac-90a3-41f0-8fcf-d3005c4e7a16 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:51.029190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1562a63-ce27-478e-b7ba-db8ceed6cf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:51.030158 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1562a63-ce27-478e-b7ba-db8ceed6cf06 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:51.030589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1562a63-ce27-478e-b7ba-db8ceed6cf06 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:51.031051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1562a63-ce27-478e-b7ba-db8ceed6cf06 None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:51.031947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1562a63-ce27-478e-b7ba-db8ceed6cf06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:51.033504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1562a63-ce27-478e-b7ba-db8ceed6cf06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:51.035741 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1562a63-ce27-478e-b7ba-db8ceed6cf06 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:51.051701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 306/609] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:51.057453 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1562a63-ce27-478e-b7ba-db8ceed6cf06 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:51.079252 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:51.173954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c096ac-90a3-41f0-8fcf-d3005c4e7a16 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:51.195323 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1562a63-ce27-478e-b7ba-db8ceed6cf06 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:51.235253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75a85e-391f-42a2-b16f-f94cb7c3eefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:51.235700 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441d59e-ee19-4b69-a4d4-8598abd9938d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:51.235975 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75a85e-391f-42a2-b16f-f94cb7c3eefe None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:51.236093 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441d59e-ee19-4b69-a4d4-8598abd9938d None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:51.236254 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441d59e-ee19-4b69-a4d4-8598abd9938d None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:51.236357 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75a85e-391f-42a2-b16f-f94cb7c3eefe None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:51.236508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441d59e-ee19-4b69-a4d4-8598abd9938d None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:51.236612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75a85e-391f-42a2-b16f-f94cb7c3eefe None admin] PATH_INFO: `/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/b7a51545146846cc8284b7044b32198a/roles/a9ef0098e982437fb0127827fa2cad38` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:51.238200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441d59e-ee19-4b69-a4d4-8598abd9938d None admin] RBAC: Adding query filter params (name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:51.238529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441d59e-ee19-4b69-a4d4-8598abd9938d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:51.239926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441d59e-ee19-4b69-a4d4-8598abd9938d None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:51.254350 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75a85e-391f-42a2-b16f-f94cb7c3eefe None admin] RBAC: Authorizing `identity:create_grant(project_id=161c90d34bb34925a1aaeb70c7de7eae, user_id=b7a51545146846cc8284b7044b32198a, role_id=a9ef0098e982437fb0127827fa2cad38)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:51.255847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75a85e-391f-42a2-b16f-f94cb7c3eefe None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:51.263354 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 304/610] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:05:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:51.271077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441d59e-ee19-4b69-a4d4-8598abd9938d None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:51.325308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 307/611] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:05:51 2022] PUT /identity/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/b7a51545146846cc8284b7044b32198a/roles/a9ef0098e982437fb0127827fa2cad38 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:05:51.373704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:51.426863 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441d59e-ee19-4b69-a4d4-8598abd9938d None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:51.486379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cee037-6b3c-41b0-8982-c148f2e9ad11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:51.487311 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cee037-6b3c-41b0-8982-c148f2e9ad11 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:51.487630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cee037-6b3c-41b0-8982-c148f2e9ad11 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:51.488607 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cee037-6b3c-41b0-8982-c148f2e9ad11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:51.506786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cee037-6b3c-41b0-8982-c148f2e9ad11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:51.508304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cee037-6b3c-41b0-8982-c148f2e9ad11 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:51.524761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-23cee037-6b3c-41b0-8982-c148f2e9ad11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:51.524761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:51.524761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:51.524761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:51.524761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:51.524761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:51.524761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:51.524761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:51.524761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:51.524761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:51.524761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:51.524761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:51.524761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:51.524761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:51.524761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:51.524761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:51.525780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:51.525780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:51.525780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:51.525780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:51.525780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:51.525780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:51.525780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:51.525780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:51.525780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:51.525780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:51.525780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:51.525780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:51.525780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:51.525780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:51.525780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:51.525780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:51.525780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:51.526694 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:51.526694 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:51.526694 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:51.531479 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 305/612] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:51.532768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75a85e-391f-42a2-b16f-f94cb7c3eefe None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:51.610919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:51.666856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75a85e-391f-42a2-b16f-f94cb7c3eefe None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:51.719264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ee134-622b-4890-b7e0-257d304b53ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:51.720222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ee134-622b-4890-b7e0-257d304b53ab None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:51.720688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ee134-622b-4890-b7e0-257d304b53ab None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:51.721155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ee134-622b-4890-b7e0-257d304b53ab None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:51.722554 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ee134-622b-4890-b7e0-257d304b53ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:51.723068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ee134-622b-4890-b7e0-257d304b53ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:51.724637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ee134-622b-4890-b7e0-257d304b53ab None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:51.745209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 308/613] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:51.751260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cee037-6b3c-41b0-8982-c148f2e9ad11 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:51.888282 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cee037-6b3c-41b0-8982-c148f2e9ad11 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:51.900356 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:51.935925 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b9386-3452-4d9f-ad1b-7bdc7dc3baf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:51.936576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b9386-3452-4d9f-ad1b-7bdc7dc3baf8 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:51.936855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b9386-3452-4d9f-ad1b-7bdc7dc3baf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:51.937135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b9386-3452-4d9f-ad1b-7bdc7dc3baf8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:51.952021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b9386-3452-4d9f-ad1b-7bdc7dc3baf8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:51.953502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b9386-3452-4d9f-ad1b-7bdc7dc3baf8 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:51.966485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-888b9386-3452-4d9f-ad1b-7bdc7dc3baf8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:05:51.966485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:51.966485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:51.966485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:51.966485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:51.966485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:51.966485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:51.966485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:51.966485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:51.966485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:51.966485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:51.966485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:51.966485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:51.966485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:51.966485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:51.966485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:51.967537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:51.968529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:51.968529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:51.968529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:51.968529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:51.968529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:51.968529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:51.968529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:51.968529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:51.968529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:05:51.968529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:51.968529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 306/614] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:05:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:51.975769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ee134-622b-4890-b7e0-257d304b53ab None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:51.998628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ad511-8ee7-4205-a922-82e5d8ca2d13 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:51.998940 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ad511-8ee7-4205-a922-82e5d8ca2d13 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:51.999211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ad511-8ee7-4205-a922-82e5d8ca2d13 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:52.000559 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 307/615] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:52.016844 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5999e-8707-4b18-9ce8-172912dba2d4 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:52.017204 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5999e-8707-4b18-9ce8-172912dba2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:52.017336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5999e-8707-4b18-9ce8-172912dba2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:52.051956 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-71e5999e-8707-4b18-9ce8-172912dba2d4 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:52.053170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-71e5999e-8707-4b18-9ce8-172912dba2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:52.120280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ee134-622b-4890-b7e0-257d304b53ab None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:52.143706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:52.145759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 308/616] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:05:52.152287 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18046d-afd8-4ed0-90bd-dee9f739da0f None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:52.152632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18046d-afd8-4ed0-90bd-dee9f739da0f None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:52.152920 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18046d-afd8-4ed0-90bd-dee9f739da0f None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:52.154503 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 309/617] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:52.165965 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1404835-e96f-4683-8e14-7059841bcc63 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:52.170002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1404835-e96f-4683-8e14-7059841bcc63 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:52.170002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1404835-e96f-4683-8e14-7059841bcc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:52.177290 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82be51f-fdfe-40e1-adbe-18c69c27dd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:52.177971 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82be51f-fdfe-40e1-adbe-18c69c27dd58 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:52.178304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82be51f-fdfe-40e1-adbe-18c69c27dd58 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:52.178580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82be51f-fdfe-40e1-adbe-18c69c27dd58 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:52.179546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82be51f-fdfe-40e1-adbe-18c69c27dd58 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:52.179850 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82be51f-fdfe-40e1-adbe-18c69c27dd58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:52.181559 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82be51f-fdfe-40e1-adbe-18c69c27dd58 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:52.204386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 309/618] 10.222.0.59 () {56 vars in 1135 bytes} [Wed Aug 31 19:05:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:52.211493 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a1404835-e96f-4683-8e14-7059841bcc63 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:52.211894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82be51f-fdfe-40e1-adbe-18c69c27dd58 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:52.213349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a1404835-e96f-4683-8e14-7059841bcc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:52.353395 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82be51f-fdfe-40e1-adbe-18c69c27dd58 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:52.397339 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 310/619] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:52.398930 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552c271-9e2c-440d-85a4-cac9322efc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:52.399639 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552c271-9e2c-440d-85a4-cac9322efc46 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:52.399923 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552c271-9e2c-440d-85a4-cac9322efc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:52.400203 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552c271-9e2c-440d-85a4-cac9322efc46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:52.424071 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1552c271-9e2c-440d-85a4-cac9322efc46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:52.426208 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1552c271-9e2c-440d-85a4-cac9322efc46 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:52.447616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-1552c271-9e2c-440d-85a4-cac9322efc46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:52.447616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:52.447616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:52.447616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:52.447616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:52.447616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:52.447616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:52.447616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:52.447616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:52.447616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:52.447616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:52.447616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:52.447616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:52.447616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:52.447616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:52.447616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:52.448877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:52.448877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:52.448877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:52.448877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:52.448877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:52.448877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:52.448877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:52.448877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:52.448877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:52.448877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:52.448877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:52.448877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:52.448877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:52.448877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:52.448877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:52.448877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:52.448877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:52.449779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:52.449779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:52.449779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:52.449779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 310/620] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:52.454814 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1404835-e96f-4683-8e14-7059841bcc63 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:52.459713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552c271-9e2c-440d-85a4-cac9322efc46 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:52.512022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:52.530903 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:52.632115 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552c271-9e2c-440d-85a4-cac9322efc46 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:52.672381 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257f77d-1336-4686-b2c1-ff166dfaa682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:52.673386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257f77d-1336-4686-b2c1-ff166dfaa682 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:52.673871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257f77d-1336-4686-b2c1-ff166dfaa682 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:52.674347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257f77d-1336-4686-b2c1-ff166dfaa682 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:52.676950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257f77d-1336-4686-b2c1-ff166dfaa682 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:52.677676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257f77d-1336-4686-b2c1-ff166dfaa682 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:52.679475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257f77d-1336-4686-b2c1-ff166dfaa682 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:52.694366 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 311/621] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:52.702098 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1404835-e96f-4683-8e14-7059841bcc63 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:52.704230 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257f77d-1336-4686-b2c1-ff166dfaa682 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:52.765507 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9115ab-e304-49e4-bb36-a70c2b443195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:52.766040 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9115ab-e304-49e4-bb36-a70c2b443195 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:52.766320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9115ab-e304-49e4-bb36-a70c2b443195 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:52.766593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9115ab-e304-49e4-bb36-a70c2b443195 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:52.779229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9115ab-e304-49e4-bb36-a70c2b443195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:52.787774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-9d9115ab-e304-49e4-bb36-a70c2b443195 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:52.787774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:52.787774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:52.787774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:52.787774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:52.787774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:52.787774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:52.787774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:52.787774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:52.787774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:52.787774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:52.787774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:52.787774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:52.787774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:52.787774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:52.787774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:52.788793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:52.788793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:52.788793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:52.788793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:52.788793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:52.788793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:52.788793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:52.788793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:52.788793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:52.788793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:52.788793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:52.788793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:52.788793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:52.788793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:52.788793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:52.788793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:52.789681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:52.789681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:52.789681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:52.789681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:52.789681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:52.789681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:52.789681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:52.789681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:52.789681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:52.789681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:52.789681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:52.789681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:52.790360 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 311/622] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:52.791428 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9115ab-e304-49e4-bb36-a70c2b443195 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:52.830806 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:52.870297 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257f77d-1336-4686-b2c1-ff166dfaa682 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:52.895690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:52.906618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9115ab-e304-49e4-bb36-a70c2b443195 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:52.920030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8265c-60f3-4cae-98a4-fc213e9445a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:52.920673 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8265c-60f3-4cae-98a4-fc213e9445a3 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:52.920990 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8265c-60f3-4cae-98a4-fc213e9445a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:52.921314 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8265c-60f3-4cae-98a4-fc213e9445a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:52.933535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8265c-60f3-4cae-98a4-fc213e9445a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:52.935024 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8265c-60f3-4cae-98a4-fc213e9445a3 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:52.951333 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47763598-4ed5-45d8-8250-3128bf5fc138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:52.951998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-9cd8265c-60f3-4cae-98a4-fc213e9445a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:52.951998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:52.951998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:52.951998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:52.951998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:52.951998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:52.951998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:52.951998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:52.951998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:52.951998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:52.951998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:52.951998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:52.951998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:52.951998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:52.951998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:52.952902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:52.952902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:52.952902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:52.952902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:52.952902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:52.952902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:52.952902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:52.952902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:52.952902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:52.952902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:52.952902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:52.952902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:52.952902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:52.952902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:52.952902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:52.952902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:52.952902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:52.953834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:52.953834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:52.953834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:52.953834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:52.953834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:52.953834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:52.953834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:52.953834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:52.953834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 312/623] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:52.954341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8265c-60f3-4cae-98a4-fc213e9445a3 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:52.955416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47763598-4ed5-45d8-8250-3128bf5fc138 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:52.955902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47763598-4ed5-45d8-8250-3128bf5fc138 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:52.956442 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47763598-4ed5-45d8-8250-3128bf5fc138 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:52.968181 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47763598-4ed5-45d8-8250-3128bf5fc138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:52.973772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-47763598-4ed5-45d8-8250-3128bf5fc138 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:52.973772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:52.973772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:52.973772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:52.973772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:52.973772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:52.973772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:52.973772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:52.973772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:52.973772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:52.973772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:52.973772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:52.973772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:52.973772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:52.973772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:52.973772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:52.974751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:52.974751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:52.974751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:52.974751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:52.974751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:52.974751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:52.974751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:52.974751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:52.974751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:52.974751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:52.974751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:52.974751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:52.974751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:52.974751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:52.974751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:52.974751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:52.975682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:52.975682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:52.975682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:52.975682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:52.975682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:52.975682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:52.975682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:52.975682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:52.975682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:52.975682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:52.975682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:52.975682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:52.978173 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 312/624] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:52.979269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47763598-4ed5-45d8-8250-3128bf5fc138 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:53.113155 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47763598-4ed5-45d8-8250-3128bf5fc138 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:53.115847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8265c-60f3-4cae-98a4-fc213e9445a3 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:53.161740 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17318bd8-cc3c-457e-9414-068302ed6648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:53.162747 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17318bd8-cc3c-457e-9414-068302ed6648 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:53.163272 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17318bd8-cc3c-457e-9414-068302ed6648 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:53.163748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17318bd8-cc3c-457e-9414-068302ed6648 None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:53.164701 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17318bd8-cc3c-457e-9414-068302ed6648 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:53.165212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17318bd8-cc3c-457e-9414-068302ed6648 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:53.166890 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17318bd8-cc3c-457e-9414-068302ed6648 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:53.167824 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0cf8e-c774-4700-84f3-a01031c4bb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:53.168441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0cf8e-c774-4700-84f3-a01031c4bb94 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:53.168719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0cf8e-c774-4700-84f3-a01031c4bb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:53.168990 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0cf8e-c774-4700-84f3-a01031c4bb94 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:53.169727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0cf8e-c774-4700-84f3-a01031c4bb94 None admin] RBAC: Adding query filter params (name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:53.170291 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0cf8e-c774-4700-84f3-a01031c4bb94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:53.172320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0cf8e-c774-4700-84f3-a01031c4bb94 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:53.183731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 313/625] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:53.184127 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 313/626] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:05:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:53.185161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17318bd8-cc3c-457e-9414-068302ed6648 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:53.190992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0cf8e-c774-4700-84f3-a01031c4bb94 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:53.202347 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:53.207321 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:53.313635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17318bd8-cc3c-457e-9414-068302ed6648 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:53.319363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0cf8e-c774-4700-84f3-a01031c4bb94 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:53.359637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ded9d-32ec-44dd-96a8-dcb389cd9c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:53.360507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ded9d-32ec-44dd-96a8-dcb389cd9c72 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:53.360943 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ded9d-32ec-44dd-96a8-dcb389cd9c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:53.361352 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ded9d-32ec-44dd-96a8-dcb389cd9c72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:53.363492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f81edd-7df8-419f-addb-f4b7227fe51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:53.364289 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f81edd-7df8-419f-addb-f4b7227fe51c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:53.364733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f81edd-7df8-419f-addb-f4b7227fe51c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:53.365154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f81edd-7df8-419f-addb-f4b7227fe51c None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:53.365980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f81edd-7df8-419f-addb-f4b7227fe51c None admin] RBAC: Adding query filter params (name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:53.366522 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f81edd-7df8-419f-addb-f4b7227fe51c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:53.368150 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f81edd-7df8-419f-addb-f4b7227fe51c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:53.375333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ded9d-32ec-44dd-96a8-dcb389cd9c72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:53.377025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ded9d-32ec-44dd-96a8-dcb389cd9c72 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:53.378801 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 314/627] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:05:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:53.379888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f81edd-7df8-419f-addb-f4b7227fe51c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:53.395146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-f22ded9d-32ec-44dd-96a8-dcb389cd9c72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:53.395146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:53.395146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:53.395146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:53.395146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:53.395146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:53.395146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:53.395146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:53.395146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:53.395146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:53.395146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:53.395146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:53.395146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:53.395146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:53.395146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:53.395146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:53.396449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:53.396449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:53.396449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:53.396449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:53.396449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:53.396449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:53.396449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:53.396449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:53.396449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:53.396449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:53.396449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:53.396449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:53.396449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:53.396449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:53.396449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:53.396449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:53.396449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:53.397393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:53.397393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:53.397393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:53.402001 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 314/628] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:53.403002 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ded9d-32ec-44dd-96a8-dcb389cd9c72 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:53.466452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:53.481977 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:53.501730 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f81edd-7df8-419f-addb-f4b7227fe51c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:53.542107 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ded9d-32ec-44dd-96a8-dcb389cd9c72 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:53.549721 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedde46f-3674-46df-a5d5-59217002f22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:53.550744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedde46f-3674-46df-a5d5-59217002f22d None admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:53.551402 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedde46f-3674-46df-a5d5-59217002f22d None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:53.551402 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedde46f-3674-46df-a5d5-59217002f22d None admin] PATH_INFO: `/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/25ed58369ffd46e8b7fa1880662926e8/roles/a9ef0098e982437fb0127827fa2cad38` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:53.568404 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedde46f-3674-46df-a5d5-59217002f22d None admin] RBAC: Authorizing `identity:create_grant(project_id=161c90d34bb34925a1aaeb70c7de7eae, user_id=25ed58369ffd46e8b7fa1880662926e8, role_id=a9ef0098e982437fb0127827fa2cad38)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:53.570606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedde46f-3674-46df-a5d5-59217002f22d None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:53.581776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15c7d3-9322-48e2-94c4-b51733cadf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:53.588067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15c7d3-9322-48e2-94c4-b51733cadf8a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:53.588067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15c7d3-9322-48e2-94c4-b51733cadf8a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:53.588067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15c7d3-9322-48e2-94c4-b51733cadf8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:53.594818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15c7d3-9322-48e2-94c4-b51733cadf8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:53.596363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15c7d3-9322-48e2-94c4-b51733cadf8a None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:53.611734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-5a15c7d3-9322-48e2-94c4-b51733cadf8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:53.611734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:53.611734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:53.611734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:53.611734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:53.611734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:53.611734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:53.611734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:53.611734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:53.611734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:53.611734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:53.611734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:53.611734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:53.611734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:53.611734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:53.611734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:53.617860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:53.617860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:53.617860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:53.617860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:53.617860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:53.617860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:53.617860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:53.617860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:53.617860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:53.617860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:53.617860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:53.617860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:53.617860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:53.617860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:53.617860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:53.617860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:53.617860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:53.618971 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:53.618971 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:53.618971 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:53.618971 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 315/629] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:53.618971 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15c7d3-9322-48e2-94c4-b51733cadf8a None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:53.644862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 315/630] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:05:53 2022] PUT /identity/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/25ed58369ffd46e8b7fa1880662926e8/roles/a9ef0098e982437fb0127827fa2cad38 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:05:53.646352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41a676-21e9-4200-97dd-f5f644cfb758 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:53.646741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41a676-21e9-4200-97dd-f5f644cfb758 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:53.647111 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41a676-21e9-4200-97dd-f5f644cfb758 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:53.648715 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 316/631] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:53.653127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41a676-21e9-4200-97dd-f5f644cfb758 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:53.778269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15c7d3-9322-48e2-94c4-b51733cadf8a None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:53.788862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41a676-21e9-4200-97dd-f5f644cfb758 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:53.794706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:53.817876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:53.822742 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71ee92-6635-4957-8d9e-e7acee276342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:53.823454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71ee92-6635-4957-8d9e-e7acee276342 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:53.823731 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71ee92-6635-4957-8d9e-e7acee276342 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:53.824003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71ee92-6635-4957-8d9e-e7acee276342 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:53.824641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71ee92-6635-4957-8d9e-e7acee276342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:53.824947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71ee92-6635-4957-8d9e-e7acee276342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:53.826308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71ee92-6635-4957-8d9e-e7acee276342 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:53.829902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b012da-f738-48a4-9b2a-dc781d5ba7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:53.830524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b012da-f738-48a4-9b2a-dc781d5ba7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:53.830803 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b012da-f738-48a4-9b2a-dc781d5ba7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:53.831311 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b012da-f738-48a4-9b2a-dc781d5ba7f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:53.831899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b012da-f738-48a4-9b2a-dc781d5ba7f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:53.832202 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b012da-f738-48a4-9b2a-dc781d5ba7f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:53.833624 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b012da-f738-48a4-9b2a-dc781d5ba7f3 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:53.840272 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 316/632] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:53.842933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85419b37-e305-471f-9fb6-bf4ebc982f00 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:53.842933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85419b37-e305-471f-9fb6-bf4ebc982f00 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:53.842933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85419b37-e305-471f-9fb6-bf4ebc982f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:53.846149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 317/633] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:53.847915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6bf57-cf6d-4556-aab8-50513aa07fb4 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:53.848203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6bf57-cf6d-4556-aab8-50513aa07fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:53.848458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6bf57-cf6d-4556-aab8-50513aa07fb4 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:53.849892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 318/634] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:53.850630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6bf57-cf6d-4556-aab8-50513aa07fb4 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:53.864863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-85419b37-e305-471f-9fb6-bf4ebc982f00 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:53.866320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-85419b37-e305-471f-9fb6-bf4ebc982f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:53.939535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 317/635] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:53.941132 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0755c-112a-4545-9ef9-af840657abc5 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:53.941643 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0755c-112a-4545-9ef9-af840657abc5 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:53.942109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0755c-112a-4545-9ef9-af840657abc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:53.961500 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-0dd0755c-112a-4545-9ef9-af840657abc5 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:53.962685 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-0dd0755c-112a-4545-9ef9-af840657abc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:53.974832 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6bf57-cf6d-4556-aab8-50513aa07fb4 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:54.020762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f272e-5160-478a-af7b-e7414d329655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:54.022246 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f272e-5160-478a-af7b-e7414d329655 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:54.023074 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f272e-5160-478a-af7b-e7414d329655 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:54.023549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f272e-5160-478a-af7b-e7414d329655 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:54.035564 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f272e-5160-478a-af7b-e7414d329655 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:54.036614 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 318/636] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:54.037278 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0755c-112a-4545-9ef9-af840657abc5 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:54.038136 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f272e-5160-478a-af7b-e7414d329655 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:54.055080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-6f8f272e-5160-478a-af7b-e7414d329655 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 19:05:54.055080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:54.055080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:54.055080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:54.055080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:54.055080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:54.055080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:54.055080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:54.055080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:54.055080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:54.055080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:54.055080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:54.055080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:54.055080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:54.055080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:54.055080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:54.056109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:54.057112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:54.057112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:54.057112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:54.057112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:54.057112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:54.057112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:54.057112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:54.057112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:54.057112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 19:05:54.057112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:54.060283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 319/637] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:54.065902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f272e-5160-478a-af7b-e7414d329655 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:54.073490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:54.123988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:54.173225 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0755c-112a-4545-9ef9-af840657abc5 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:54.211061 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb08d7e-386b-4728-ac82-8dd54ef62b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:54.212486 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb08d7e-386b-4728-ac82-8dd54ef62b29 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:54.212966 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb08d7e-386b-4728-ac82-8dd54ef62b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:54.213431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb08d7e-386b-4728-ac82-8dd54ef62b29 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:54.221637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb08d7e-386b-4728-ac82-8dd54ef62b29 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:54.223535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb08d7e-386b-4728-ac82-8dd54ef62b29 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:54.236031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-2eb08d7e-386b-4728-ac82-8dd54ef62b29 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:05:54.236031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:54.236031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:54.236031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:54.236031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:54.236031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:54.236031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:54.236031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:54.236031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:54.236031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:54.236031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:54.236031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:54.236031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:54.236031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:54.236031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:54.236031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:54.237819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:54.238776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:54.238776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:54.238776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:54.238776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:54.238776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:54.238776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:54.238776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:54.238776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:54.238776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:05:54.238776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:54.241408 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 319/638] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 31 19:05:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:54.246036 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f272e-5160-478a-af7b-e7414d329655 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:54.247185 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb08d7e-386b-4728-ac82-8dd54ef62b29 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:54.300835 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b828a09-9df4-430b-bd12-80f5463839d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:54.301459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b828a09-9df4-430b-bd12-80f5463839d4 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:54.301744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b828a09-9df4-430b-bd12-80f5463839d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:54.302020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b828a09-9df4-430b-bd12-80f5463839d4 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:54.303847 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b828a09-9df4-430b-bd12-80f5463839d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:54.304095 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b828a09-9df4-430b-bd12-80f5463839d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:54.305662 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b828a09-9df4-430b-bd12-80f5463839d4 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:54.338909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 320/639] 10.222.0.59 () {56 vars in 1137 bytes} [Wed Aug 31 19:05:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:54.338909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b828a09-9df4-430b-bd12-80f5463839d4 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:54.379541 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:54.388339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb08d7e-386b-4728-ac82-8dd54ef62b29 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:54.423393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba947e4b-22e5-4dcd-87a8-a6ed133b0439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:54.424008 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba947e4b-22e5-4dcd-87a8-a6ed133b0439 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:54.424285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba947e4b-22e5-4dcd-87a8-a6ed133b0439 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:54.424555 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba947e4b-22e5-4dcd-87a8-a6ed133b0439 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:54.426391 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba947e4b-22e5-4dcd-87a8-a6ed133b0439 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:54.426698 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba947e4b-22e5-4dcd-87a8-a6ed133b0439 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:54.428491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba947e4b-22e5-4dcd-87a8-a6ed133b0439 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:54.450781 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 320/640] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 31 19:05:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:54.450781 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba947e4b-22e5-4dcd-87a8-a6ed133b0439 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:54.451936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:54.454770 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b828a09-9df4-430b-bd12-80f5463839d4 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:54.493902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc3189-03f2-4fa0-97d6-115c9a11229c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:54.495270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc3189-03f2-4fa0-97d6-115c9a11229c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:54.495565 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc3189-03f2-4fa0-97d6-115c9a11229c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:54.495822 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc3189-03f2-4fa0-97d6-115c9a11229c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:54.507391 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc3189-03f2-4fa0-97d6-115c9a11229c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:54.512153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-9ecc3189-03f2-4fa0-97d6-115c9a11229c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:54.512153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:54.512153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:54.512153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:54.512153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:54.512153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:54.512153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:54.512153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:54.512153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:54.512153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:54.512153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:54.512153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:54.512153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:54.512153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:54.512153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:54.512153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:54.513104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:54.513104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:54.513104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:54.513104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:54.513104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:54.513104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:54.513104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:54.513104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:54.513104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:54.513104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:54.513104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:54.513104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:54.513104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:54.513104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:54.513104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:54.513104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:54.513971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:54.513971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:54.513971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:54.513971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:54.513971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:54.513971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:54.513971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:54.513971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:54.513971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:54.513971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:54.513971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:54.513971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:54.513971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 321/641] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:54.513971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc3189-03f2-4fa0-97d6-115c9a11229c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:54.556357 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba947e4b-22e5-4dcd-87a8-a6ed133b0439 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:54.591240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f3c49-32a8-44ca-a4e2-16f274211fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:54.591844 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f3c49-32a8-44ca-a4e2-16f274211fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:54.592120 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f3c49-32a8-44ca-a4e2-16f274211fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:54.592384 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f3c49-32a8-44ca-a4e2-16f274211fd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:54.610441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:54.627436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc3189-03f2-4fa0-97d6-115c9a11229c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:54.667298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f7b69-dde4-4903-b28e-f55cbda87000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:54.667507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f3c49-32a8-44ca-a4e2-16f274211fd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:54.667934 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f7b69-dde4-4903-b28e-f55cbda87000 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:54.668270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f7b69-dde4-4903-b28e-f55cbda87000 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:54.668543 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f7b69-dde4-4903-b28e-f55cbda87000 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:54.668788 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f3c49-32a8-44ca-a4e2-16f274211fd7 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:54.680099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f7b69-dde4-4903-b28e-f55cbda87000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:54.685329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-850f7b69-dde4-4903-b28e-f55cbda87000 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:54.685329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:54.685329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:54.685329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:54.685329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:54.685329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:54.685329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:54.685329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:54.685329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:54.685329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:54.685329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:54.685329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:54.685329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:54.685329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:54.685329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:54.685329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:54.686361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:54.686361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:54.686361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:54.686361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:54.686361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:54.686361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:54.686361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:54.686361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:54.686361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:54.686361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:54.686361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:54.686361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:54.686361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:54.686361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:54.686361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:54.686361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:54.687622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:54.687622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:54.687622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:54.687622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:54.687622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:54.687622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:54.687622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:54.687622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:54.687622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:54.687622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:54.687622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:54.687622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:54.687622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 322/642] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:54.687622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f7b69-dde4-4903-b28e-f55cbda87000 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:54.711585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:54.783519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 321/643] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:54.783519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f3c49-32a8-44ca-a4e2-16f274211fd7 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:54.807763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f7b69-dde4-4903-b28e-f55cbda87000 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:54.834431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:54.848108 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b0ffb-22e4-49fe-9bd3-f0f710ef5f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:54.848816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b0ffb-22e4-49fe-9bd3-f0f710ef5f6b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:54.849097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b0ffb-22e4-49fe-9bd3-f0f710ef5f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:54.849382 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b0ffb-22e4-49fe-9bd3-f0f710ef5f6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:54.889295 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f3c49-32a8-44ca-a4e2-16f274211fd7 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:54.926433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957f7b9-a0cc-49b1-b8b7-1db5203df377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:54.927107 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957f7b9-a0cc-49b1-b8b7-1db5203df377 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:54.927384 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957f7b9-a0cc-49b1-b8b7-1db5203df377 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:54.927652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957f7b9-a0cc-49b1-b8b7-1db5203df377 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:54.927767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b0ffb-22e4-49fe-9bd3-f0f710ef5f6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:54.928303 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957f7b9-a0cc-49b1-b8b7-1db5203df377 None admin] RBAC: Adding query filter params (name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:54.928601 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957f7b9-a0cc-49b1-b8b7-1db5203df377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:54.929144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b0ffb-22e4-49fe-9bd3-f0f710ef5f6b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:54.929928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957f7b9-a0cc-49b1-b8b7-1db5203df377 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:54.943051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 322/644] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:05:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:54.944015 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957f7b9-a0cc-49b1-b8b7-1db5203df377 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:54.949797 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:55.040339 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 323/645] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:55.040339 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b0ffb-22e4-49fe-9bd3-f0f710ef5f6b None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:55.057466 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957f7b9-a0cc-49b1-b8b7-1db5203df377 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:55.072626 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:55.096012 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24168a05-653d-4783-80e9-520c4ec736dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:55.096629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24168a05-653d-4783-80e9-520c4ec736dd None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:55.096928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24168a05-653d-4783-80e9-520c4ec736dd None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:55.097209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24168a05-653d-4783-80e9-520c4ec736dd None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:55.097883 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24168a05-653d-4783-80e9-520c4ec736dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:55.098189 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24168a05-653d-4783-80e9-520c4ec736dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:55.100435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24168a05-653d-4783-80e9-520c4ec736dd None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:55.114905 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 323/646] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:05:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:55.114905 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24168a05-653d-4783-80e9-520c4ec736dd None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:55.158691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b0ffb-22e4-49fe-9bd3-f0f710ef5f6b None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:55.186166 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:55.202290 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb1f5e-0bae-49e1-a187-a081e1714c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:55.206947 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb1f5e-0bae-49e1-a187-a081e1714c27 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:55.206947 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb1f5e-0bae-49e1-a187-a081e1714c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:55.206947 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb1f5e-0bae-49e1-a187-a081e1714c27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:55.215465 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb1f5e-0bae-49e1-a187-a081e1714c27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:55.217227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb1f5e-0bae-49e1-a187-a081e1714c27 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:55.219540 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24168a05-653d-4783-80e9-520c4ec736dd None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:55.234899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-3ecb1f5e-0bae-49e1-a187-a081e1714c27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:55.234899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:55.234899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:55.234899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:55.234899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:55.234899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:55.234899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:55.234899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:55.234899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:55.234899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:55.234899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:55.234899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:55.234899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:55.234899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:55.234899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:55.234899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:55.236030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:55.236030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:55.236030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:55.236030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:55.236030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:55.236030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:55.236030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:55.236030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:55.236030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:55.236030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:55.236030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:55.236030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:55.236030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:55.236030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:55.236030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:55.236030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:55.236030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:55.237896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:55.237896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:55.237896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:55.237896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 324/647] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:55.237896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb1f5e-0bae-49e1-a187-a081e1714c27 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:55.259235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f0291-a591-4d68-9747-2f95d2c63dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:55.259867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f0291-a591-4d68-9747-2f95d2c63dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:55.260141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f0291-a591-4d68-9747-2f95d2c63dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:55.260408 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f0291-a591-4d68-9747-2f95d2c63dd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:55.275069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f0291-a591-4d68-9747-2f95d2c63dd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:55.276908 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f0291-a591-4d68-9747-2f95d2c63dd4 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:55.297280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-6a9f0291-a591-4d68-9747-2f95d2c63dd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:55.297280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:55.297280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:55.297280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:55.297280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:55.297280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:55.297280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:55.297280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:55.297280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:55.297280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:55.297280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:55.297280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:55.297280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:55.297280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:55.297280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:55.297280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:55.298432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:55.298432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:55.298432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:55.298432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:55.298432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:55.298432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:55.298432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:55.298432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:55.298432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:55.298432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:55.298432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:55.298432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:55.298432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:55.298432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:55.298432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:55.298432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:55.298432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:55.299576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:55.299576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:55.299576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:55.301130 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 324/648] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:55.302089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f0291-a591-4d68-9747-2f95d2c63dd4 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:55.331587 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:55.360320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb1f5e-0bae-49e1-a187-a081e1714c27 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:55.398569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da96b0-5d74-4e70-963b-e099cedbfaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:55.399505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da96b0-5d74-4e70-963b-e099cedbfaf7 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:55.400181 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da96b0-5d74-4e70-963b-e099cedbfaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:55.400463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da96b0-5d74-4e70-963b-e099cedbfaf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:55.412163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da96b0-5d74-4e70-963b-e099cedbfaf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:55.412597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f0291-a591-4d68-9747-2f95d2c63dd4 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:55.413609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da96b0-5d74-4e70-963b-e099cedbfaf7 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:55.428991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-b5da96b0-5d74-4e70-963b-e099cedbfaf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:55.428991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:55.428991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:55.428991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:55.428991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:55.428991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:55.428991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:55.428991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:55.428991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:55.428991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:55.428991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:55.428991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:55.428991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:55.428991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:55.428991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:55.428991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:55.430014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:55.430014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:55.430014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:55.430014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:55.430014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:55.430014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:55.430014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:55.430014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:55.430014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:55.430014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:55.430014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:55.430014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:55.430014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:55.430014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:55.430014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:55.430014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:55.430014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:55.431055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:55.431055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:55.431055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:55.431055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 325/649] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:55.431055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da96b0-5d74-4e70-963b-e099cedbfaf7 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:55.453701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ca8f4-9830-4d21-9734-2e82c100d6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:55.454486 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ca8f4-9830-4d21-9734-2e82c100d6df None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:55.454898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ca8f4-9830-4d21-9734-2e82c100d6df None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:55.455284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ca8f4-9830-4d21-9734-2e82c100d6df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:55.459616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:55.467816 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ca8f4-9830-4d21-9734-2e82c100d6df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:55.469317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ca8f4-9830-4d21-9734-2e82c100d6df None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:55.485091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-0e2ca8f4-9830-4d21-9734-2e82c100d6df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:55.485091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:55.485091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:55.485091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:55.485091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:55.485091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:55.485091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:55.485091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:55.485091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:55.485091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:55.485091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:55.485091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:55.485091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:55.485091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:55.485091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:55.485091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:55.486113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:55.486113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:55.486113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:55.486113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:55.486113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:55.486113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:55.486113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:55.486113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:55.486113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:55.486113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:55.486113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:55.486113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:55.486113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:55.486113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:55.486113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:55.486113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:55.486113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:55.488178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:55.488178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:55.488178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:55.488178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 325/650] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:55.488178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ca8f4-9830-4d21-9734-2e82c100d6df None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:55.544418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da96b0-5d74-4e70-963b-e099cedbfaf7 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:55.573242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:55.582630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab9749-7371-4990-bc3f-231de2b1f944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:55.584613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab9749-7371-4990-bc3f-231de2b1f944 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:55.584613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab9749-7371-4990-bc3f-231de2b1f944 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:55.584613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab9749-7371-4990-bc3f-231de2b1f944 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:55.584613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab9749-7371-4990-bc3f-231de2b1f944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:55.584890 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab9749-7371-4990-bc3f-231de2b1f944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:55.586049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab9749-7371-4990-bc3f-231de2b1f944 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:55.595676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ca8f4-9830-4d21-9734-2e82c100d6df None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:55.598269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 326/651] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:55.599116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab9749-7371-4990-bc3f-231de2b1f944 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:55.635835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096af3c-e407-42fd-81e3-0cb8d17a825d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:55.636460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096af3c-e407-42fd-81e3-0cb8d17a825d None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:55.636744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096af3c-e407-42fd-81e3-0cb8d17a825d None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:55.637021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096af3c-e407-42fd-81e3-0cb8d17a825d None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:55.637675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096af3c-e407-42fd-81e3-0cb8d17a825d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:55.637984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096af3c-e407-42fd-81e3-0cb8d17a825d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:55.639413 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096af3c-e407-42fd-81e3-0cb8d17a825d None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:55.656352 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 326/652] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:55.657198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096af3c-e407-42fd-81e3-0cb8d17a825d None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:55.699483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:55.714935 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab9749-7371-4990-bc3f-231de2b1f944 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:55.749115 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68138a-f83c-41d0-8e4e-66fa46ad69cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:55.749734 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68138a-f83c-41d0-8e4e-66fa46ad69cb None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:55.750019 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68138a-f83c-41d0-8e4e-66fa46ad69cb None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:55.750296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68138a-f83c-41d0-8e4e-66fa46ad69cb None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:55.750977 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68138a-f83c-41d0-8e4e-66fa46ad69cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:55.751288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68138a-f83c-41d0-8e4e-66fa46ad69cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:55.752660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68138a-f83c-41d0-8e4e-66fa46ad69cb None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:55.761624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096af3c-e407-42fd-81e3-0cb8d17a825d None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:55.764225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 327/653] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:55.765038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68138a-f83c-41d0-8e4e-66fa46ad69cb None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:55.797752 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036eb61-ad53-413e-8373-8f5bea16f2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:55.798363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036eb61-ad53-413e-8373-8f5bea16f2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:55.798643 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036eb61-ad53-413e-8373-8f5bea16f2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:55.798952 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036eb61-ad53-413e-8373-8f5bea16f2b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:55.799592 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036eb61-ad53-413e-8373-8f5bea16f2b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:55.800457 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036eb61-ad53-413e-8373-8f5bea16f2b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:55.801815 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036eb61-ad53-413e-8373-8f5bea16f2b0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:55.808271 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:55.813800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 327/654] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:55.814647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036eb61-ad53-413e-8373-8f5bea16f2b0 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:55.868511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68138a-f83c-41d0-8e4e-66fa46ad69cb None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:55.903077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb66386-f7f1-49f2-a2ae-a90d2dedf2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:55.903718 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb66386-f7f1-49f2-a2ae-a90d2dedf2ec None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:55.903998 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb66386-f7f1-49f2-a2ae-a90d2dedf2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:55.904278 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb66386-f7f1-49f2-a2ae-a90d2dedf2ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:55.911605 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb66386-f7f1-49f2-a2ae-a90d2dedf2ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:55.913056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb66386-f7f1-49f2-a2ae-a90d2dedf2ec None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:55.919663 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036eb61-ad53-413e-8373-8f5bea16f2b0 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:55.924379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-9bb66386-f7f1-49f2-a2ae-a90d2dedf2ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:55.924379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:55.924379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:55.924379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:55.924379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:55.924379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:55.924379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:55.924379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:55.924379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:55.924379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:55.924379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:55.924379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:55.924379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:55.924379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:55.924379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:55.925437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:55.925437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:55.925437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:55.925437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:55.925437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:55.925437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:55.925437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:55.925437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:55.925437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:55.925437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:55.925437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:55.925437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:55.925437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:55.925437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:55.925437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:55.925437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:55.925437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:55.926355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:55.926355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:55.926355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:55.926355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:55.926355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:55.926355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:55.926355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:55.926355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:55.926355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 328/655] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:55.926857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b2606-128b-4e2f-a047-ef85f0003a1d None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:55.927010 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b2606-128b-4e2f-a047-ef85f0003a1d None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:55.927292 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b2606-128b-4e2f-a047-ef85f0003a1d None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:55.928670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 329/656] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:55.929276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b2606-128b-4e2f-a047-ef85f0003a1d None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:55.933941 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:55.960618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecc7a2-4173-46fe-bfd2-9e6f18fe6bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:55.961253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecc7a2-4173-46fe-bfd2-9e6f18fe6bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:55.961546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecc7a2-4173-46fe-bfd2-9e6f18fe6bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:55.961821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecc7a2-4173-46fe-bfd2-9e6f18fe6bb5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:55.969530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecc7a2-4173-46fe-bfd2-9e6f18fe6bb5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:55.970762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecc7a2-4173-46fe-bfd2-9e6f18fe6bb5 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:55.982512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-d3ecc7a2-4173-46fe-bfd2-9e6f18fe6bb5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:05:55.982512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:55.982512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:55.982512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:55.982512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:55.982512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:55.982512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:55.982512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:55.982512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:55.982512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:55.982512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:55.982512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:55.982512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:55.982512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:55.982512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:55.982512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:55.983662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:55.984841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:55.984841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:55.984841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:55.984841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:55.984841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:55.984841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:55.984841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:55.984841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:55.984841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:05:55.984841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:55.988740 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 328/657] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:55.989563 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecc7a2-4173-46fe-bfd2-9e6f18fe6bb5 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:56.035262 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b2606-128b-4e2f-a047-ef85f0003a1d None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:56.047882 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:56.069280 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e472aa7-3553-4118-b9da-990a5d397cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:56.069918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e472aa7-3553-4118-b9da-990a5d397cdb None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:56.070203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e472aa7-3553-4118-b9da-990a5d397cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:56.070640 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e472aa7-3553-4118-b9da-990a5d397cdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:56.077433 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e472aa7-3553-4118-b9da-990a5d397cdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:56.078901 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e472aa7-3553-4118-b9da-990a5d397cdb None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:56.090851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecc7a2-4173-46fe-bfd2-9e6f18fe6bb5 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:56.091034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-3e472aa7-3553-4118-b9da-990a5d397cdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:56.091034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:56.091034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:56.091034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:56.091034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:56.091034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:56.091034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:56.091034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:56.091034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:56.091034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:56.091034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:56.091034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:56.091034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:56.091034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:56.091034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:56.092056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:56.092056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:56.092056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:56.092056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:56.092056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:56.092056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:56.092056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:56.092056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:56.092056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:56.092056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:56.092056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:56.092056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:56.092056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:56.092056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:56.092056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:56.092056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:56.092056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:56.093189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:56.093189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:56.093189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:56.093189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:56.093189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:56.093189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:56.093189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:56.093189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:56.093189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 330/658] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:56.093778 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e472aa7-3553-4118-b9da-990a5d397cdb None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:56.130043 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a029a-37f8-4d2c-9c9b-adb55cf77f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:56.130602 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a029a-37f8-4d2c-9c9b-adb55cf77f21 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:56.131071 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a029a-37f8-4d2c-9c9b-adb55cf77f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:56.131349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a029a-37f8-4d2c-9c9b-adb55cf77f21 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:56.138840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a029a-37f8-4d2c-9c9b-adb55cf77f21 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:56.140306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a029a-37f8-4d2c-9c9b-adb55cf77f21 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:56.152443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-0c6a029a-37f8-4d2c-9c9b-adb55cf77f21 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 19:05:56.152443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:56.152443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:56.152443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:56.152443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:56.152443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:56.152443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:56.152443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:56.152443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:56.152443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:56.152443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:56.152443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:56.152443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:56.152443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:56.152443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:56.152443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:56.158133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:56.159422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:56.159422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:56.159422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:56.159422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:56.159422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:56.159422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:56.159422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:56.159422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:56.159422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 19:05:56.159422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:56.159422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 329/659] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:56.159422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf9eeb-c03f-44e6-8183-c51b2bb66f05 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:56.159422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf9eeb-c03f-44e6-8183-c51b2bb66f05 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:56.160323 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf9eeb-c03f-44e6-8183-c51b2bb66f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:56.160478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:56.175692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ddbf9eeb-c03f-44e6-8183-c51b2bb66f05 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:56.176878 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ddbf9eeb-c03f-44e6-8183-c51b2bb66f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:56.199931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e472aa7-3553-4118-b9da-990a5d397cdb None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:56.234227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198aecfa-1d4b-44ec-a64c-d77bfa48dda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:56.234831 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198aecfa-1d4b-44ec-a64c-d77bfa48dda2 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:56.235143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198aecfa-1d4b-44ec-a64c-d77bfa48dda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:56.235421 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198aecfa-1d4b-44ec-a64c-d77bfa48dda2 None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:56.236104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198aecfa-1d4b-44ec-a64c-d77bfa48dda2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:56.236412 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198aecfa-1d4b-44ec-a64c-d77bfa48dda2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:56.238135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198aecfa-1d4b-44ec-a64c-d77bfa48dda2 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:56.249319 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 330/660] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:56.255146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 331/661] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:56.255277 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf9eeb-c03f-44e6-8183-c51b2bb66f05 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:56.255358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198aecfa-1d4b-44ec-a64c-d77bfa48dda2 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:56.315907 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:56.367962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198aecfa-1d4b-44ec-a64c-d77bfa48dda2 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:56.371030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf9eeb-c03f-44e6-8183-c51b2bb66f05 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:56.394608 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:56.406698 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd5607-f997-4312-a8e7-43be00cbf687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:56.407171 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c995f56-c163-442d-88e1-90fc5843b635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:56.407319 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd5607-f997-4312-a8e7-43be00cbf687 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:56.407511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd5607-f997-4312-a8e7-43be00cbf687 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:56.407763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd5607-f997-4312-a8e7-43be00cbf687 None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:56.407862 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c995f56-c163-442d-88e1-90fc5843b635 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:56.408042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c995f56-c163-442d-88e1-90fc5843b635 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:56.408315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c995f56-c163-442d-88e1-90fc5843b635 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:56.408480 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd5607-f997-4312-a8e7-43be00cbf687 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:56.408765 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd5607-f997-4312-a8e7-43be00cbf687 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:56.409066 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c995f56-c163-442d-88e1-90fc5843b635 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:56.409374 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c995f56-c163-442d-88e1-90fc5843b635 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:56.410457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd5607-f997-4312-a8e7-43be00cbf687 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:56.410843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c995f56-c163-442d-88e1-90fc5843b635 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:56.422630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 332/662] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:56.423515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd5607-f997-4312-a8e7-43be00cbf687 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:56.433592 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 331/663] 10.222.0.59 () {56 vars in 1137 bytes} [Wed Aug 31 19:05:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:56.434720 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c995f56-c163-442d-88e1-90fc5843b635 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:56.540645 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c995f56-c163-442d-88e1-90fc5843b635 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:56.540858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd5607-f997-4312-a8e7-43be00cbf687 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:56.548055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:56.573943 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27355ac5-205c-4e8c-a37e-d17fd8b8e570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:56.574543 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27355ac5-205c-4e8c-a37e-d17fd8b8e570 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:56.574817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27355ac5-205c-4e8c-a37e-d17fd8b8e570 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:56.575117 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27355ac5-205c-4e8c-a37e-d17fd8b8e570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:56.575824 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27355ac5-205c-4e8c-a37e-d17fd8b8e570 None admin] RBAC: Adding query filter params (role.id=a9ef0098e982437fb0127827fa2cad38, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=f281d778f1404984b4d778a1c26aef1d) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:56.576128 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27355ac5-205c-4e8c-a37e-d17fd8b8e570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:56.577161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bf341-8305-4362-9040-2b1af000307c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:56.577616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27355ac5-205c-4e8c-a37e-d17fd8b8e570 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:56.577774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bf341-8305-4362-9040-2b1af000307c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:56.578163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bf341-8305-4362-9040-2b1af000307c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:56.578446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bf341-8305-4362-9040-2b1af000307c None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:56.579219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545bf341-8305-4362-9040-2b1af000307c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:56.579528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545bf341-8305-4362-9040-2b1af000307c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:56.581923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545bf341-8305-4362-9040-2b1af000307c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:56.584124 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 332/664] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:56 2022] GET /identity/v3/role_assignments?user.id=f281d778f1404984b4d778a1c26aef1d&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=a9ef0098e982437fb0127827fa2cad38 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:56.584904 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27355ac5-205c-4e8c-a37e-d17fd8b8e570 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:56.602305 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 333/665] 10.222.0.59 () {56 vars in 1139 bytes} [Wed Aug 31 19:05:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:56.603595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bf341-8305-4362-9040-2b1af000307c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:56.665127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:56.715301 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27355ac5-205c-4e8c-a37e-d17fd8b8e570 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:56.731254 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bf341-8305-4362-9040-2b1af000307c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:56.759644 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7aeed-f647-44fd-a0f9-913bca4141df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:56.760261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7aeed-f647-44fd-a0f9-913bca4141df None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:56.760534 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7aeed-f647-44fd-a0f9-913bca4141df None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:56.760804 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7aeed-f647-44fd-a0f9-913bca4141df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:56.761558 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7aeed-f647-44fd-a0f9-913bca4141df None admin] RBAC: Adding query filter params (role.id=a9ef0098e982437fb0127827fa2cad38, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=d461422ce98442bcbe6b254a8dad5e8a) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:56.761825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7aeed-f647-44fd-a0f9-913bca4141df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:56.763303 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7aeed-f647-44fd-a0f9-913bca4141df None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:56.771150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 333/666] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:56 2022] GET /identity/v3/role_assignments?user.id=d461422ce98442bcbe6b254a8dad5e8a&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=a9ef0098e982437fb0127827fa2cad38 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:56.772053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7aeed-f647-44fd-a0f9-913bca4141df None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:56.781341 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b1b3a-7709-45af-be8d-583257e46417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:56.782340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b1b3a-7709-45af-be8d-583257e46417 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:56.782457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b1b3a-7709-45af-be8d-583257e46417 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:56.782641 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b1b3a-7709-45af-be8d-583257e46417 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:56.821511 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:56.883321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b1b3a-7709-45af-be8d-583257e46417 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:56.885356 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b1b3a-7709-45af-be8d-583257e46417 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:56.912724 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7aeed-f647-44fd-a0f9-913bca4141df None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:56.951026 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:56.969308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2428e4c-7b76-470f-94f8-5778f36a6f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:56.969962 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2428e4c-7b76-470f-94f8-5778f36a6f0f None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:56.970245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2428e4c-7b76-470f-94f8-5778f36a6f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:56.970521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2428e4c-7b76-470f-94f8-5778f36a6f0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:56.984735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2428e4c-7b76-470f-94f8-5778f36a6f0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:56.991692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-c2428e4c-7b76-470f-94f8-5778f36a6f0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:56.991692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:56.991692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:56.991692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:56.991692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:56.991692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:56.991692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:56.991692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:56.991692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:56.991692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:56.991692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:56.991692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:56.991692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:56.991692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:56.991692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:56.991692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:56.992889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:56.992889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:56.992889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:56.992889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:56.992889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:56.992889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:56.992889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:56.992889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:56.992889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:56.992889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:56.992889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:56.992889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:56.992889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:56.992889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:56.992889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:56.992889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:56.993959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:56.993959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:56.993959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:56.993959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:56.993959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:56.993959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:56.993959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:56.993959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:56.993959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:56.993959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:56.993959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:56.993959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:56.995485 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 334/667] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:56.996301 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2428e4c-7b76-470f-94f8-5778f36a6f0f None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:57.048325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 334/668] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:57.049404 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b1b3a-7709-45af-be8d-583257e46417 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:57.129417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:57.137836 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2428e4c-7b76-470f-94f8-5778f36a6f0f None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:57.177820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fe868-ef01-47b4-a9aa-3ab02da883d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:57.178600 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fe868-ef01-47b4-a9aa-3ab02da883d3 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:57.179015 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fe868-ef01-47b4-a9aa-3ab02da883d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:57.179409 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fe868-ef01-47b4-a9aa-3ab02da883d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:57.181689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b1b3a-7709-45af-be8d-583257e46417 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:57.233035 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104fb96-5251-454f-bbfa-e1751053bc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:57.233881 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104fb96-5251-454f-bbfa-e1751053bc02 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:57.234554 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104fb96-5251-454f-bbfa-e1751053bc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:57.235002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104fb96-5251-454f-bbfa-e1751053bc02 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:57.236263 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104fb96-5251-454f-bbfa-e1751053bc02 None admin] RBAC: Adding query filter params (name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:57.237026 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104fb96-5251-454f-bbfa-e1751053bc02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:57.239528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104fb96-5251-454f-bbfa-e1751053bc02 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:57.261747 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 335/669] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:05:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:57.261747 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104fb96-5251-454f-bbfa-e1751053bc02 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:57.266001 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:57.266959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9fe868-ef01-47b4-a9aa-3ab02da883d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:57.270623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9fe868-ef01-47b4-a9aa-3ab02da883d3 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:57.396563 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:57.410914 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 335/670] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:57.410914 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fe868-ef01-47b4-a9aa-3ab02da883d3 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:57.429099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104fb96-5251-454f-bbfa-e1751053bc02 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:57.480842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219e75f-6d6f-4e3b-8d34-6b8962ca23c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:57.481781 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219e75f-6d6f-4e3b-8d34-6b8962ca23c7 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:57.482072 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219e75f-6d6f-4e3b-8d34-6b8962ca23c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:57.482351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219e75f-6d6f-4e3b-8d34-6b8962ca23c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:57.501135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219e75f-6d6f-4e3b-8d34-6b8962ca23c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:57.504211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219e75f-6d6f-4e3b-8d34-6b8962ca23c7 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:57.528321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-1219e75f-6d6f-4e3b-8d34-6b8962ca23c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:57.528321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:57.528321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:57.528321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:57.528321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:57.528321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:57.528321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:57.528321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:57.528321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:57.528321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:57.528321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:57.528321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:57.528321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:57.528321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:57.528321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:57.528321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:57.529996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:57.529996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:57.529996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:57.529996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:57.529996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:57.529996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:57.529996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:57.529996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:57.529996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:57.529996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:57.529996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:57.529996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:57.529996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:57.529996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:57.529996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:57.529996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:57.529996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:57.531609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:57.531609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:57.531609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:57.534758 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 336/671] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:57.537065 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219e75f-6d6f-4e3b-8d34-6b8962ca23c7 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:57.554343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fe868-ef01-47b4-a9aa-3ab02da883d3 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:57.608371 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15170c1-11b9-423c-972d-70a7090f05c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:57.609081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15170c1-11b9-423c-972d-70a7090f05c3 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:57.609398 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15170c1-11b9-423c-972d-70a7090f05c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:57.609672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15170c1-11b9-423c-972d-70a7090f05c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:57.623228 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:57.625317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15170c1-11b9-423c-972d-70a7090f05c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:57.627171 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15170c1-11b9-423c-972d-70a7090f05c3 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:57.651985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-a15170c1-11b9-423c-972d-70a7090f05c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:57.651985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:57.651985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:57.651985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:57.651985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:57.651985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:57.651985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:57.651985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:57.651985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:57.651985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:57.651985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:57.651985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:57.651985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:57.651985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:57.651985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:57.651985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:57.653025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:57.653025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:57.653025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:57.653025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:57.653025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:57.653025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:57.653025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:57.653025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:57.653025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:57.653025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:57.653025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:57.653025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:57.653025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:57.653025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:57.653025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:57.653025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:57.653025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:57.654025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:57.654025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:57.654025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:57.657050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 336/672] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:57.658237 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15170c1-11b9-423c-972d-70a7090f05c3 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:57.687297 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219e75f-6d6f-4e3b-8d34-6b8962ca23c7 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:57.725966 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:57.735382 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f85ab-9c0c-48e4-b122-09d04a578549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:57.736032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f85ab-9c0c-48e4-b122-09d04a578549 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:57.736313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f85ab-9c0c-48e4-b122-09d04a578549 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:57.736655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f85ab-9c0c-48e4-b122-09d04a578549 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:57.751630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f85ab-9c0c-48e4-b122-09d04a578549 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:57.753041 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f85ab-9c0c-48e4-b122-09d04a578549 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:57.768816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-db0f85ab-9c0c-48e4-b122-09d04a578549 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:57.768816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:57.768816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:57.768816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:57.768816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:57.768816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:57.768816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:57.768816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:57.768816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:57.768816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:57.768816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:57.768816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:57.768816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:57.768816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:57.768816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:57.768816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:57.770116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:57.770116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:57.770116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:57.770116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:57.770116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:57.770116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:57.770116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:57.770116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:57.770116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:57.770116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:57.770116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:57.770116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:57.770116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:57.770116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:57.770116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:57.770116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:57.770116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:57.771075 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:57.771075 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:57.771075 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:57.771075 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 337/673] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:57.771559 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f85ab-9c0c-48e4-b122-09d04a578549 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:57.815435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15170c1-11b9-423c-972d-70a7090f05c3 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:57.865561 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd3e86-4f02-41dd-b5e1-e0f6c0810e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:57.866598 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd3e86-4f02-41dd-b5e1-e0f6c0810e21 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:57.867135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd3e86-4f02-41dd-b5e1-e0f6c0810e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:57.867456 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd3e86-4f02-41dd-b5e1-e0f6c0810e21 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:57.868119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd3e86-4f02-41dd-b5e1-e0f6c0810e21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:57.868433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd3e86-4f02-41dd-b5e1-e0f6c0810e21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:57.869834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd3e86-4f02-41dd-b5e1-e0f6c0810e21 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:57.887479 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 337/674] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:57.891136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd3e86-4f02-41dd-b5e1-e0f6c0810e21 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:57.911703 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f85ab-9c0c-48e4-b122-09d04a578549 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:57.927440 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:57.964603 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca7fdd-29eb-48a2-a82c-c825b7b3b61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:57.965606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca7fdd-29eb-48a2-a82c-c825b7b3b61f None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:57.966084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca7fdd-29eb-48a2-a82c-c825b7b3b61f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:57.966555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca7fdd-29eb-48a2-a82c-c825b7b3b61f None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:57.967907 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca7fdd-29eb-48a2-a82c-c825b7b3b61f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:57.969370 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca7fdd-29eb-48a2-a82c-c825b7b3b61f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:57.971161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca7fdd-29eb-48a2-a82c-c825b7b3b61f None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:57.989487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 338/675] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:57.990915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca7fdd-29eb-48a2-a82c-c825b7b3b61f None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:58.033704 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd3e86-4f02-41dd-b5e1-e0f6c0810e21 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:58.044469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:58.083544 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c706d-f7ae-49ee-8a96-aba7f2be00f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:58.084563 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c706d-f7ae-49ee-8a96-aba7f2be00f4 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:58.085056 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c706d-f7ae-49ee-8a96-aba7f2be00f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:58.085541 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c706d-f7ae-49ee-8a96-aba7f2be00f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:58.086633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c706d-f7ae-49ee-8a96-aba7f2be00f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:58.087194 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c706d-f7ae-49ee-8a96-aba7f2be00f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:58.089156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c706d-f7ae-49ee-8a96-aba7f2be00f4 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:58.104149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 338/676] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:58.105388 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c706d-f7ae-49ee-8a96-aba7f2be00f4 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:58.145505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca7fdd-29eb-48a2-a82c-c825b7b3b61f None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:58.201307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b42ff8-7284-4ebc-94a7-e7faae36d72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:58.202372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b42ff8-7284-4ebc-94a7-e7faae36d72a None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:58.202944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b42ff8-7284-4ebc-94a7-e7faae36d72a None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:58.203225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b42ff8-7284-4ebc-94a7-e7faae36d72a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:58.212575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b42ff8-7284-4ebc-94a7-e7faae36d72a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:58.214418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b42ff8-7284-4ebc-94a7-e7faae36d72a None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:58.228450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-08b42ff8-7284-4ebc-94a7-e7faae36d72a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:58.228450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:58.228450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:58.228450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:58.228450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:58.228450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:58.228450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:58.228450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:58.228450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:58.228450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:58.228450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:58.228450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:58.228450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:58.228450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:58.228450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:58.230223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:58.230223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:58.230223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:58.230223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:58.230223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:58.230223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:58.230223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:58.230223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:58.230223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:58.230223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:58.230223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:58.230223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:58.230223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:58.230223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:58.230223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:58.230223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:58.230223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:58.231722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:58.231722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:58.231722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:58.231722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:58.231722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:58.231722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:58.231722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:58.231722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:58.231722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 339/677] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:58.231722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b42ff8-7284-4ebc-94a7-e7faae36d72a None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:58.275733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:58.279373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c706d-f7ae-49ee-8a96-aba7f2be00f4 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:58.324720 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af600e19-26c6-4a06-9c7f-ec78e42f3ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:58.325635 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af600e19-26c6-4a06-9c7f-ec78e42f3ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:58.326060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af600e19-26c6-4a06-9c7f-ec78e42f3ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:58.326540 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af600e19-26c6-4a06-9c7f-ec78e42f3ba0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:58.336213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af600e19-26c6-4a06-9c7f-ec78e42f3ba0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:58.338248 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af600e19-26c6-4a06-9c7f-ec78e42f3ba0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:58.354583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-af600e19-26c6-4a06-9c7f-ec78e42f3ba0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:05:58.354583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:58.354583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:58.354583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:58.354583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:58.354583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:58.354583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:58.354583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:58.354583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:58.354583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:58.354583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:58.354583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:58.354583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:58.354583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:58.354583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:58.354583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:58.355604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:58.356565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:58.356565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:58.356565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:58.356565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:58.356565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:58.356565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:58.356565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:58.356565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:58.356565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:05:58.356565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:58.364278 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 339/678] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:05:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:58.365455 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af600e19-26c6-4a06-9c7f-ec78e42f3ba0 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:58.395710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b42ff8-7284-4ebc-94a7-e7faae36d72a None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:58.399509 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:58.434978 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d961b4-3574-466c-8233-ac1de8a49115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:58.436154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d961b4-3574-466c-8233-ac1de8a49115 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:58.436683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d961b4-3574-466c-8233-ac1de8a49115 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:58.437174 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d961b4-3574-466c-8233-ac1de8a49115 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:58.445912 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d961b4-3574-466c-8233-ac1de8a49115 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:58.448070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d961b4-3574-466c-8233-ac1de8a49115 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:58.462407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-b0d961b4-3574-466c-8233-ac1de8a49115 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:58.462407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:58.462407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:58.462407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:58.462407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:58.462407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:58.462407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:58.462407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:58.462407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:58.462407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:58.462407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:58.462407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:58.462407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:58.462407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:58.462407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:58.463591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:58.463591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:58.463591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:58.463591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:58.463591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:58.463591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:58.463591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:58.463591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:58.463591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:58.463591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:58.463591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:58.463591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:58.463591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:58.463591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:58.463591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:58.463591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:58.463591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:58.464683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:58.464683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:58.464683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:58.464683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:58.464683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:58.464683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:58.464683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:58.464683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:58.472481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 340/679] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:58.473889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d961b4-3574-466c-8233-ac1de8a49115 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:58.499307 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af600e19-26c6-4a06-9c7f-ec78e42f3ba0 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:58.541593 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345e261-051f-4c12-98b7-a63a643859da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:58.542597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345e261-051f-4c12-98b7-a63a643859da None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:58.543135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345e261-051f-4c12-98b7-a63a643859da None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:58.543709 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345e261-051f-4c12-98b7-a63a643859da None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:58.544620 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345e261-051f-4c12-98b7-a63a643859da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:58.545140 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345e261-051f-4c12-98b7-a63a643859da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:58.546796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345e261-051f-4c12-98b7-a63a643859da None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:58.561158 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 340/680] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:58.562598 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345e261-051f-4c12-98b7-a63a643859da None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:58.597923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:58.611444 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d961b4-3574-466c-8233-ac1de8a49115 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:58.650751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74fde81-bf71-4bd8-980a-7115fa04d554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:58.651458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74fde81-bf71-4bd8-980a-7115fa04d554 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:58.651704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74fde81-bf71-4bd8-980a-7115fa04d554 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:58.651933 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74fde81-bf71-4bd8-980a-7115fa04d554 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:58.652676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74fde81-bf71-4bd8-980a-7115fa04d554 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:58.653344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74fde81-bf71-4bd8-980a-7115fa04d554 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:58.654520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74fde81-bf71-4bd8-980a-7115fa04d554 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:58.663293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:58.680978 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 341/681] 10.222.0.59 () {56 vars in 1135 bytes} [Wed Aug 31 19:05:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:58.680978 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74fde81-bf71-4bd8-980a-7115fa04d554 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:58.686337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345e261-051f-4c12-98b7-a63a643859da None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:58.724704 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9a8eb-deb5-43d3-82d2-0df355abebb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:58.725747 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9a8eb-deb5-43d3-82d2-0df355abebb3 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:58.726283 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9a8eb-deb5-43d3-82d2-0df355abebb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:58.726769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9a8eb-deb5-43d3-82d2-0df355abebb3 None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:58.727788 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9a8eb-deb5-43d3-82d2-0df355abebb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:58.728416 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9a8eb-deb5-43d3-82d2-0df355abebb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:58.730140 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9a8eb-deb5-43d3-82d2-0df355abebb3 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:58.747919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 341/682] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:58.749279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9a8eb-deb5-43d3-82d2-0df355abebb3 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:58.790131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74fde81-bf71-4bd8-980a-7115fa04d554 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:58.830002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc8cfe-4680-4c40-a2d1-d1d662b46fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:58.830620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc8cfe-4680-4c40-a2d1-d1d662b46fed None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:58.831534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc8cfe-4680-4c40-a2d1-d1d662b46fed None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:58.832038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc8cfe-4680-4c40-a2d1-d1d662b46fed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:58.832858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc8cfe-4680-4c40-a2d1-d1d662b46fed None admin] RBAC: Adding query filter params (role.id=a9ef0098e982437fb0127827fa2cad38, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=108a698aa3e6477b9ae896d68f22bd0c) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:58.833204 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc8cfe-4680-4c40-a2d1-d1d662b46fed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:58.834749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc8cfe-4680-4c40-a2d1-d1d662b46fed None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:58.841851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 342/683] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:58 2022] GET /identity/v3/role_assignments?user.id=108a698aa3e6477b9ae896d68f22bd0c&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=a9ef0098e982437fb0127827fa2cad38 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:58.842695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc8cfe-4680-4c40-a2d1-d1d662b46fed None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:58.858630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:58.863183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9a8eb-deb5-43d3-82d2-0df355abebb3 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:58.901669 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723f75e-fffe-445f-a91b-bc31d30a325b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:58.902428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723f75e-fffe-445f-a91b-bc31d30a325b None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:58.902817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723f75e-fffe-445f-a91b-bc31d30a325b None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:58.903251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723f75e-fffe-445f-a91b-bc31d30a325b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:58.921692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:58.968702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc8cfe-4680-4c40-a2d1-d1d662b46fed None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:58.982909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723f75e-fffe-445f-a91b-bc31d30a325b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:58.984651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723f75e-fffe-445f-a91b-bc31d30a325b None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:59.009960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82289d-694d-48f8-af6e-45fc2d727468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:59.010911 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82289d-694d-48f8-af6e-45fc2d727468 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:59.011611 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82289d-694d-48f8-af6e-45fc2d727468 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:59.012332 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82289d-694d-48f8-af6e-45fc2d727468 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:59.014019 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82289d-694d-48f8-af6e-45fc2d727468 None admin] RBAC: Adding query filter params (role.id=a9ef0098e982437fb0127827fa2cad38, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=b7a51545146846cc8284b7044b32198a) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:59.014771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82289d-694d-48f8-af6e-45fc2d727468 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:59.016996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82289d-694d-48f8-af6e-45fc2d727468 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:59.025939 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 343/684] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:58 2022] GET /identity/v3/role_assignments?user.id=b7a51545146846cc8284b7044b32198a&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=a9ef0098e982437fb0127827fa2cad38 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:59.027681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361982d-8c11-43fb-aab1-4f4c45ae6eb2 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:59.028136 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361982d-8c11-43fb-aab1-4f4c45ae6eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:59.028546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361982d-8c11-43fb-aab1-4f4c45ae6eb2 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:59.030427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 344/685] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:59.036863 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0fb4f-564c-408e-b754-fe725c47f302 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:59.037430 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0fb4f-564c-408e-b754-fe725c47f302 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:59.037865 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0fb4f-564c-408e-b754-fe725c47f302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:59.072061 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-89a0fb4f-564c-408e-b754-fe725c47f302 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:59.073305 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-89a0fb4f-564c-408e-b754-fe725c47f302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:59.155841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 342/686] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:05:59.157177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4a13e-6e41-4686-966c-12421d35ec31 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:59.157471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4a13e-6e41-4686-966c-12421d35ec31 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:59.157739 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4a13e-6e41-4686-966c-12421d35ec31 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:59.160192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 343/687] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:59.160948 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4a13e-6e41-4686-966c-12421d35ec31 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:59.179917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 345/688] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:05:59.181771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5485a5-6137-4518-a17e-e54a1623144d None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:59.182770 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5485a5-6137-4518-a17e-e54a1623144d None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:59.183617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5485a5-6137-4518-a17e-e54a1623144d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:59.193910 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:59.209840 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-bb5485a5-6137-4518-a17e-e54a1623144d None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:59.211829 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-bb5485a5-6137-4518-a17e-e54a1623144d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:59.224011 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:59.310066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 346/689] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:05:59.323642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4a13e-6e41-4686-966c-12421d35ec31 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:59.364164 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e994bb0-0912-4c37-ba1a-097eab0b0d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:59.365088 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e994bb0-0912-4c37-ba1a-097eab0b0d5f None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:59.365521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e994bb0-0912-4c37-ba1a-097eab0b0d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:59.365954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e994bb0-0912-4c37-ba1a-097eab0b0d5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:59.377192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e994bb0-0912-4c37-ba1a-097eab0b0d5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:59.380119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e994bb0-0912-4c37-ba1a-097eab0b0d5f None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:59.397460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-4e994bb0-0912-4c37-ba1a-097eab0b0d5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:59.397460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:59.397460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:59.397460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:59.397460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:59.397460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:59.397460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:59.397460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:59.397460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:59.397460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:59.397460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:59.397460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:59.397460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:59.397460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:59.397460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:59.397460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:59.398757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:59.398757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:59.398757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:59.398757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:59.398757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:59.398757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:59.398757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:59.398757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:59.398757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:59.398757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:59.398757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:59.398757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:59.398757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:59.398757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:59.398757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:59.398757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:59.398757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:59.399961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:59.399961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:59.399961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:59.401824 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 344/690] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:59.407260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5485a5-6137-4518-a17e-e54a1623144d None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:59.483590 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e994bb0-0912-4c37-ba1a-097eab0b0d5f None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:59.559869 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5485a5-6137-4518-a17e-e54a1623144d None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:59.598898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2fbd1-6328-47c3-ab92-62c8a334322f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:59.598898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2fbd1-6328-47c3-ab92-62c8a334322f None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:59.598898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2fbd1-6328-47c3-ab92-62c8a334322f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:59.598898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2fbd1-6328-47c3-ab92-62c8a334322f None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:59.599388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2fbd1-6328-47c3-ab92-62c8a334322f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:59.599388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2fbd1-6328-47c3-ab92-62c8a334322f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:59.600624 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2fbd1-6328-47c3-ab92-62c8a334322f None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:59.602375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:59.614995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 347/691] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:59.620247 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2fbd1-6328-47c3-ab92-62c8a334322f None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:59.698325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:59.701161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e994bb0-0912-4c37-ba1a-097eab0b0d5f None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:59.738722 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310267d-9a93-40bb-bc2c-94deff0957b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:59.739575 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310267d-9a93-40bb-bc2c-94deff0957b7 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:59.739698 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310267d-9a93-40bb-bc2c-94deff0957b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:59.739925 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310267d-9a93-40bb-bc2c-94deff0957b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:59.750630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2fbd1-6328-47c3-ab92-62c8a334322f None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:59.751197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310267d-9a93-40bb-bc2c-94deff0957b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:59.757996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-6310267d-9a93-40bb-bc2c-94deff0957b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:59.757996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:59.757996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:59.757996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:59.757996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:59.757996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:59.757996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:59.757996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:59.757996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:59.757996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:59.757996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:59.757996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:59.757996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:59.757996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:59.757996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:59.757996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:59.759222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:59.759222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:59.759222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:59.759222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:59.759222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:59.759222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:59.759222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:59.759222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:59.759222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:59.759222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:59.759222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:59.759222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:59.759222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:59.759222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:59.759222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:59.759222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:59.760246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:59.760246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:59.760246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:59.760246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:59.760246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:59.760246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:59.760246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:59.760246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:59.760246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:59.760246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:59.760246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:59.760246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:59.760246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 345/692] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:05:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:59.760992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310267d-9a93-40bb-bc2c-94deff0957b7 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:59.801794 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3cd9e7-bf7e-4c7a-9c84-1b9a7240b487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:59.802779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cd9e7-bf7e-4c7a-9c84-1b9a7240b487 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:59.803282 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cd9e7-bf7e-4c7a-9c84-1b9a7240b487 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:59.803758 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cd9e7-bf7e-4c7a-9c84-1b9a7240b487 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:59.811158 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3cd9e7-bf7e-4c7a-9c84-1b9a7240b487 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:59.812937 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3cd9e7-bf7e-4c7a-9c84-1b9a7240b487 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:59.830464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-5a3cd9e7-bf7e-4c7a-9c84-1b9a7240b487 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:59.830464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:59.830464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:59.830464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:59.830464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:59.830464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:59.830464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:59.830464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:59.830464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:59.830464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:59.830464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:59.830464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:59.830464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:59.830464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:59.830464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:59.831846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:59.831846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:59.831846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:59.831846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:59.831846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:59.831846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:59.831846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:59.831846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:59.831846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:59.831846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:59.831846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:59.831846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:59.831846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:59.831846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:59.831846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:59.831846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:59.831846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:59.832855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:59.832855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:59.832855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:59.832855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:59.832855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:59.832855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:59.832855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:59.832855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:05:59.836197 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 348/693] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:59.837499 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3cd9e7-bf7e-4c7a-9c84-1b9a7240b487 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:59.894416 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310267d-9a93-40bb-bc2c-94deff0957b7 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:59.899963 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:59.933439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecbc75-6cc5-4c78-9889-040ed4e20cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:59.934421 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecbc75-6cc5-4c78-9889-040ed4e20cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:59.934919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecbc75-6cc5-4c78-9889-040ed4e20cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:59.935376 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecbc75-6cc5-4c78-9889-040ed4e20cb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:59.948508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3cd9e7-bf7e-4c7a-9c84-1b9a7240b487 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:59.948948 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceecbc75-6cc5-4c78-9889-040ed4e20cb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:59.950746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceecbc75-6cc5-4c78-9889-040ed4e20cb8 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:59.966833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-ceecbc75-6cc5-4c78-9889-040ed4e20cb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:59.966833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:59.966833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:59.966833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:59.966833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:59.966833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:59.966833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:59.966833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:59.966833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:59.966833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:59.966833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:59.966833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:59.966833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:59.966833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:59.966833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:59.966833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:59.967881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:59.967881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:59.967881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:59.967881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:59.967881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:59.967881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:59.967881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:59.967881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:59.967881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:59.967881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:59.967881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:59.967881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:59.967881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:59.967881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:59.967881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:59.967881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:59.967881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:59.968857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:59.968857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:59.968857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:05:59.970679 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 346/694] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:59.972421 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecbc75-6cc5-4c78-9889-040ed4e20cb8 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:59.986092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:59.997788 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae79ff-a8f5-4f61-82ef-0821d64fa439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:59.998672 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae79ff-a8f5-4f61-82ef-0821d64fa439 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:59.999171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae79ff-a8f5-4f61-82ef-0821d64fa439 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:59.999595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae79ff-a8f5-4f61-82ef-0821d64fa439 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:00.000411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae79ff-a8f5-4f61-82ef-0821d64fa439 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:00.000876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae79ff-a8f5-4f61-82ef-0821d64fa439 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:00.002470 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae79ff-a8f5-4f61-82ef-0821d64fa439 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:00.018010 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 349/695] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:00.019928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae79ff-a8f5-4f61-82ef-0821d64fa439 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:00.085190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecbc75-6cc5-4c78-9889-040ed4e20cb8 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:00.134330 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79fe78c-9f27-4f58-b865-fa7776f6e633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:00.134994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fe78c-9f27-4f58-b865-fa7776f6e633 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:00.135273 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fe78c-9f27-4f58-b865-fa7776f6e633 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:00.135548 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fe78c-9f27-4f58-b865-fa7776f6e633 None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:00.136408 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79fe78c-9f27-4f58-b865-fa7776f6e633 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:00.136717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79fe78c-9f27-4f58-b865-fa7776f6e633 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:00.138486 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79fe78c-9f27-4f58-b865-fa7776f6e633 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:00.149440 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae79ff-a8f5-4f61-82ef-0821d64fa439 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:00.154981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 347/696] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:00.156675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79fe78c-9f27-4f58-b865-fa7776f6e633 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:00.202243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1feb0-7f73-45de-8ab3-8e34010a4edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:00.202789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1feb0-7f73-45de-8ab3-8e34010a4edb None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:00.203166 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1feb0-7f73-45de-8ab3-8e34010a4edb None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:00.203394 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1feb0-7f73-45de-8ab3-8e34010a4edb None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:00.204285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1feb0-7f73-45de-8ab3-8e34010a4edb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:00.204699 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1feb0-7f73-45de-8ab3-8e34010a4edb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:00.206706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1feb0-7f73-45de-8ab3-8e34010a4edb None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:00.225338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 350/697] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:00.225338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:00.225338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1feb0-7f73-45de-8ab3-8e34010a4edb None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:00.260011 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:00.297569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79fe78c-9f27-4f58-b865-fa7776f6e633 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:00.341607 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa7441-827b-4923-b15c-320a515d1910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:00.343102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa7441-827b-4923-b15c-320a515d1910 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:00.343587 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa7441-827b-4923-b15c-320a515d1910 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:00.344057 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa7441-827b-4923-b15c-320a515d1910 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:00.356871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa7441-827b-4923-b15c-320a515d1910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:00.358701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa7441-827b-4923-b15c-320a515d1910 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:00.361357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1feb0-7f73-45de-8ab3-8e34010a4edb None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:00.376946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-44fa7441-827b-4923-b15c-320a515d1910 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:00.376946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:00.376946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:00.376946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:00.376946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:00.376946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:00.376946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:00.376946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:00.376946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:00.376946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:00.376946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:00.376946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:00.376946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:00.376946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:00.376946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:00.376946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:00.378855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:00.378855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:00.378855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:00.378855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:00.378855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:00.378855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:00.378855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:00.378855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:00.378855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:00.378855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:00.378855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:00.378855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:00.378855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:00.378855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:00.378855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:00.378855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:00.378855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:00.379867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:00.379867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:00.379867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:00.381664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 348/698] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:00.382471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa7441-827b-4923-b15c-320a515d1910 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:00.405914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604f6c9-1438-45b1-8430-f5076e073d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:00.407186 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604f6c9-1438-45b1-8430-f5076e073d0c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:00.408024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604f6c9-1438-45b1-8430-f5076e073d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:00.408553 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604f6c9-1438-45b1-8430-f5076e073d0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:00.410321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604f6c9-1438-45b1-8430-f5076e073d0c None admin] RBAC: Adding query filter params (role.id=a9ef0098e982437fb0127827fa2cad38, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=25ed58369ffd46e8b7fa1880662926e8) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:00.410854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604f6c9-1438-45b1-8430-f5076e073d0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:00.413270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604f6c9-1438-45b1-8430-f5076e073d0c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:00.422485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 351/699] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:06:00 2022] GET /identity/v3/role_assignments?user.id=25ed58369ffd46e8b7fa1880662926e8&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=a9ef0098e982437fb0127827fa2cad38 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:00.423567 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604f6c9-1438-45b1-8430-f5076e073d0c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:00.475476 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:00.593911 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604f6c9-1438-45b1-8430-f5076e073d0c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:00.599424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa7441-827b-4923-b15c-320a515d1910 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:00.636191 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:00.652233 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3324e-b3b5-4041-b65d-1de88cedde33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:00.652853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3324e-b3b5-4041-b65d-1de88cedde33 None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:00.653133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3324e-b3b5-4041-b65d-1de88cedde33 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:00.653407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3324e-b3b5-4041-b65d-1de88cedde33 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:00.654063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3324e-b3b5-4041-b65d-1de88cedde33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:00.655594 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed83be-170c-4c82-8751-af87d6a11fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:00.656039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3324e-b3b5-4041-b65d-1de88cedde33 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:00.656693 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed83be-170c-4c82-8751-af87d6a11fbe None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:00.657178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed83be-170c-4c82-8751-af87d6a11fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:00.657661 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed83be-170c-4c82-8751-af87d6a11fbe None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:00.658635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed83be-170c-4c82-8751-af87d6a11fbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:00.659756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed83be-170c-4c82-8751-af87d6a11fbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:00.661623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed83be-170c-4c82-8751-af87d6a11fbe None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:00.679820 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 352/700] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:00.685144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed83be-170c-4c82-8751-af87d6a11fbe None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:00.745963 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 349/701] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:06:00 2022] POST /identity/v3/users => generated 254 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:00.798294 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:00.802386 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed83be-170c-4c82-8751-af87d6a11fbe None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:00.843638 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b7769-31e7-4345-8990-fd406b93aeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:00.844288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b7769-31e7-4345-8990-fd406b93aeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:00.844568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b7769-31e7-4345-8990-fd406b93aeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:00.844845 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b7769-31e7-4345-8990-fd406b93aeb0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:00.854061 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b7769-31e7-4345-8990-fd406b93aeb0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:00.855628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b7769-31e7-4345-8990-fd406b93aeb0 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:00.869923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-721b7769-31e7-4345-8990-fd406b93aeb0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:06:00.869923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:00.869923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:00.869923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:00.869923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:00.869923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:00.869923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:00.869923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:00.869923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:00.869923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:00.869923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:00.869923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:00.869923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:06:00.869923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:06:00.869923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:06:00.869923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:00.871067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:00.872163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:00.872163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:00.872163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:06:00.872163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:06:00.872163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:06:00.872163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:06:00.872163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:06:00.872163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:06:00.872163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:06:00.872163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:00.872764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 353/702] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 31 19:06:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:00.878075 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3324e-b3b5-4041-b65d-1de88cedde33 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:01.000385 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3324e-b3b5-4041-b65d-1de88cedde33 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:01.039124 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207e427-b6c4-432a-9ad7-b1883a40c644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:01.039786 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207e427-b6c4-432a-9ad7-b1883a40c644 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:01.040069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207e427-b6c4-432a-9ad7-b1883a40c644 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:01.040343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207e427-b6c4-432a-9ad7-b1883a40c644 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:01.041083 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207e427-b6c4-432a-9ad7-b1883a40c644 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:01.041387 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207e427-b6c4-432a-9ad7-b1883a40c644 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:01.042817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207e427-b6c4-432a-9ad7-b1883a40c644 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:01.068916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 350/703] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 31 19:06:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:01.071160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b7769-31e7-4345-8990-fd406b93aeb0 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:01.126086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d113c02-cbc0-4a1b-b8e9-989ce35b8c38 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:01.126412 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d113c02-cbc0-4a1b-b8e9-989ce35b8c38 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:01.126638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d113c02-cbc0-4a1b-b8e9-989ce35b8c38 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:01.129341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 351/704] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:01.137003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd3deb-4035-4c75-a60e-f0459f8db22a None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:01.137415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd3deb-4035-4c75-a60e-f0459f8db22a None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:01.137793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd3deb-4035-4c75-a60e-f0459f8db22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:01.161989 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-dccd3deb-4035-4c75-a60e-f0459f8db22a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:01.163571 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-dccd3deb-4035-4c75-a60e-f0459f8db22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:01.194482 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b7769-31e7-4345-8990-fd406b93aeb0 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:01.204676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:01.234997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e6c84-62c9-4eda-88fb-a131bf4cee0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:01.235756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e6c84-62c9-4eda-88fb-a131bf4cee0f None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:01.236039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e6c84-62c9-4eda-88fb-a131bf4cee0f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:01.237661 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e6c84-62c9-4eda-88fb-a131bf4cee0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:01.259411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 352/705] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:06:01.281827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:01.344714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e6c84-62c9-4eda-88fb-a131bf4cee0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:01.346203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e6c84-62c9-4eda-88fb-a131bf4cee0f None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:01.451855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 354/706] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:06:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:06:01.460055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd3deb-4035-4c75-a60e-f0459f8db22a None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:01.512914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae2ccf-8d3a-48ab-819e-630f891b8d45 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:01.513198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae2ccf-8d3a-48ab-819e-630f891b8d45 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:01.513395 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae2ccf-8d3a-48ab-819e-630f891b8d45 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:01.514751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 355/707] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:01.524115 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa9203-e148-40e4-bfe3-d9cbf455233f None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:01.524449 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa9203-e148-40e4-bfe3-d9cbf455233f None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:01.524665 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa9203-e148-40e4-bfe3-d9cbf455233f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:01.552676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-94fa9203-e148-40e4-bfe3-d9cbf455233f None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:01.553881 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-94fa9203-e148-40e4-bfe3-d9cbf455233f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:01.619795 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd3deb-4035-4c75-a60e-f0459f8db22a None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:01.658157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:01.665163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46236c01-7f82-4ea9-8362-421178193443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:01.665870 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46236c01-7f82-4ea9-8362-421178193443 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:01.666159 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46236c01-7f82-4ea9-8362-421178193443 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:01.666431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46236c01-7f82-4ea9-8362-421178193443 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:01.673321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 356/708] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:06:01.675855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa9203-e148-40e4-bfe3-d9cbf455233f None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:01.684976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46236c01-7f82-4ea9-8362-421178193443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:01.686437 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46236c01-7f82-4ea9-8362-421178193443 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:01.715033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-46236c01-7f82-4ea9-8362-421178193443 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:01.715033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:01.715033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:01.715033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:01.715033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:01.715033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:01.715033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:01.715033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:01.715033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:01.715033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:01.715033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:01.715033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:01.715033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:01.715033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:01.715033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:01.715033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:01.716068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:01.716068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:01.716068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:01.716068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:01.716068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:01.716068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:01.716068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:01.716068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:01.716068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:01.716068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:01.716068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:01.716068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:01.716068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:01.716068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:01.716068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:01.716068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:01.716068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:01.717012 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:01.717012 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:01.717012 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:01.723353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 353/709] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:01.724270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46236c01-7f82-4ea9-8362-421178193443 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:01.769732 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:01.797146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa9203-e148-40e4-bfe3-d9cbf455233f None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:01.837444 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572ffa1-326b-48ce-8c13-42095ace9ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:01.838969 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572ffa1-326b-48ce-8c13-42095ace9ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:01.839594 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572ffa1-326b-48ce-8c13-42095ace9ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:01.840080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572ffa1-326b-48ce-8c13-42095ace9ed5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:01.841122 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572ffa1-326b-48ce-8c13-42095ace9ed5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:01.842690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572ffa1-326b-48ce-8c13-42095ace9ed5 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:01.855448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-5572ffa1-326b-48ce-8c13-42095ace9ed5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:06:01.855448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:01.855448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:01.855448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:01.855448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:01.855448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:01.855448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:01.855448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:01.855448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:06:01.855448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:01.855448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:01.855448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:01.855448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:01.855448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:01.855448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:01.855448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:01.855448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:01.856550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:01.856550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:01.856550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:01.856550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:01.856550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:01.856550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:01.856550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:01.856550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:01.856550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:01.856550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:01.856550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:01.856550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:01.856550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:01.856550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:01.856550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:01.856550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:01.857425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:01.857425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:01.857425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:01.857425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:01.857425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:01.857425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:01.857425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:01.857425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:01.857425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:01.857425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:01.857425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:01.857425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:01.857425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:06:01.857425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:01.860449 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 357/710] 10.222.0.59 () {56 vars in 1129 bytes} [Wed Aug 31 19:06:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:01.866448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572ffa1-326b-48ce-8c13-42095ace9ed5 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:01.976334 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46236c01-7f82-4ea9-8362-421178193443 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:01.977881 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572ffa1-326b-48ce-8c13-42095ace9ed5 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:01.991235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:02.015090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2a0e7-ece2-462e-b9a8-e35989e3a95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:02.023037 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2a0e7-ece2-462e-b9a8-e35989e3a95d None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:02.023037 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2a0e7-ece2-462e-b9a8-e35989e3a95d None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:02.023037 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2a0e7-ece2-462e-b9a8-e35989e3a95d None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:02.023037 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2a0e7-ece2-462e-b9a8-e35989e3a95d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:02.023037 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2a0e7-ece2-462e-b9a8-e35989e3a95d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:02.023037 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2a0e7-ece2-462e-b9a8-e35989e3a95d None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:02.025586 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 358/711] 10.222.0.59 () {56 vars in 1124 bytes} [Wed Aug 31 19:06:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:02.036211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2a0e7-ece2-462e-b9a8-e35989e3a95d None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:02.041192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128f288-33c5-4a0b-b164-05e886db9096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:02.042058 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128f288-33c5-4a0b-b164-05e886db9096 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:02.042574 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128f288-33c5-4a0b-b164-05e886db9096 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:02.043106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128f288-33c5-4a0b-b164-05e886db9096 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:02.043957 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128f288-33c5-4a0b-b164-05e886db9096 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:02.044467 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128f288-33c5-4a0b-b164-05e886db9096 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:02.046044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128f288-33c5-4a0b-b164-05e886db9096 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:02.060599 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 354/712] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:02.067141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128f288-33c5-4a0b-b164-05e886db9096 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:02.164609 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:02.184731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2a0e7-ece2-462e-b9a8-e35989e3a95d None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:02.192075 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128f288-33c5-4a0b-b164-05e886db9096 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:02.204196 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:02.225703 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef202c0-b46c-4049-83ba-f8e9f2b06f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:02.226333 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef202c0-b46c-4049-83ba-f8e9f2b06f8e None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:02.226614 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef202c0-b46c-4049-83ba-f8e9f2b06f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:02.226909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef202c0-b46c-4049-83ba-f8e9f2b06f8e None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:02.227558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef202c0-b46c-4049-83ba-f8e9f2b06f8e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:02.227864 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef202c0-b46c-4049-83ba-f8e9f2b06f8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:02.229248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef202c0-b46c-4049-83ba-f8e9f2b06f8e None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:02.235910 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 359/713] 10.222.0.59 () {56 vars in 1124 bytes} [Wed Aug 31 19:06:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:02.237729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef202c0-b46c-4049-83ba-f8e9f2b06f8e None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:02.255702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59e878-6a06-4572-a5c7-252a00ce481f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:02.258916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59e878-6a06-4572-a5c7-252a00ce481f None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:02.260526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59e878-6a06-4572-a5c7-252a00ce481f None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:02.260806 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59e878-6a06-4572-a5c7-252a00ce481f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:02.275520 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee59e878-6a06-4572-a5c7-252a00ce481f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:02.276920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee59e878-6a06-4572-a5c7-252a00ce481f None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:02.300703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-ee59e878-6a06-4572-a5c7-252a00ce481f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:02.300703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:02.300703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:02.300703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:02.300703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:02.300703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:02.300703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:02.300703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:02.300703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:02.300703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:02.300703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:02.300703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:02.300703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:06:02.300703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:06:02.300703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:06:02.301777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:06:02.301777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:02.301777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:02.301777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:02.301777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:02.301777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:02.301777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:02.301777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:02.301777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:02.301777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:02.301777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:02.301777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:02.301777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:02.301777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:02.301777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:02.301777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:02.301777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:02.302917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:06:02.302917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:06:02.302917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:06:02.302917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:06:02.302917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:06:02.302917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:06:02.302917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:02.302917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:02.303474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 355/714] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:02.310071 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59e878-6a06-4572-a5c7-252a00ce481f None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:02.461044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef202c0-b46c-4049-83ba-f8e9f2b06f8e None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:02.503198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59e878-6a06-4572-a5c7-252a00ce481f None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:02.520717 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e2199-bc49-4b8b-bdb1-84e0fb1b7817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:02.521705 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e2199-bc49-4b8b-bdb1-84e0fb1b7817 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:02.522197 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e2199-bc49-4b8b-bdb1-84e0fb1b7817 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:02.522680 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e2199-bc49-4b8b-bdb1-84e0fb1b7817 None admin] PATH_INFO: `/v3/services/network` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:02.523539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e2199-bc49-4b8b-bdb1-84e0fb1b7817 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:02.525146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e2199-bc49-4b8b-bdb1-84e0fb1b7817 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:02.541337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cb36a-84ce-42c7-8444-0e303ccff41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:02.541874 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cb36a-84ce-42c7-8444-0e303ccff41b None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:02.542150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cb36a-84ce-42c7-8444-0e303ccff41b None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:02.542426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cb36a-84ce-42c7-8444-0e303ccff41b None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:02.543144 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055cb36a-84ce-42c7-8444-0e303ccff41b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:02.543449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055cb36a-84ce-42c7-8444-0e303ccff41b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:02.544853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055cb36a-84ce-42c7-8444-0e303ccff41b None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:02.548393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-c98e2199-bc49-4b8b-bdb1-84e0fb1b7817 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:06:02.548393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:02.548393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:02.548393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:02.548393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:02.548393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:02.548393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:02.548393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:02.548393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:06:02.548393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:02.548393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:02.548393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:02.548393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:02.548393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:02.548393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:02.548393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:02.548393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:02.550256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:02.550256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:02.550256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:02.550256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:02.550256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:02.550256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:02.550256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:02.550256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:02.550256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:02.550256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:02.550256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:02.550256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:02.550256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:02.550256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:02.550256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:02.550256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:02.551178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:02.551178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:02.551178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:02.551178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:02.551178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:02.551178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:02.551178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:02.551178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:02.551178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:02.551178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:02.551178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:02.551178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:02.551178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:06:02.551178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:02.552486 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:02.553845 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 360/715] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:02 2022] GET /identity/v3/services/network => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:02.559335 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 356/716] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:06:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:02.560221 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e2199-bc49-4b8b-bdb1-84e0fb1b7817 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:02.568077 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cb36a-84ce-42c7-8444-0e303ccff41b None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:02.688357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:02.702826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cb36a-84ce-42c7-8444-0e303ccff41b None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:02.743812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448eda2-6d2b-449b-b617-c12e9091b77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:02.744821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448eda2-6d2b-449b-b617-c12e9091b77e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:02.745377 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448eda2-6d2b-449b-b617-c12e9091b77e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:02.745853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448eda2-6d2b-449b-b617-c12e9091b77e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:02.746772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448eda2-6d2b-449b-b617-c12e9091b77e None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=d461422ce98442bcbe6b254a8dad5e8a) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:02.747327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448eda2-6d2b-449b-b617-c12e9091b77e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:02.749266 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448eda2-6d2b-449b-b617-c12e9091b77e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:02.757465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 357/717] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:06:02 2022] GET /identity/v3/role_assignments?user.id=d461422ce98442bcbe6b254a8dad5e8a&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:02.792560 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e2199-bc49-4b8b-bdb1-84e0fb1b7817 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:02.830300 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199e3e9-97ce-497a-a79d-73e648604286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:02.831400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199e3e9-97ce-497a-a79d-73e648604286 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:02.831913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199e3e9-97ce-497a-a79d-73e648604286 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:02.832384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199e3e9-97ce-497a-a79d-73e648604286 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:02.833262 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199e3e9-97ce-497a-a79d-73e648604286 None admin] RBAC: Adding query filter params (name=network) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:02.833774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199e3e9-97ce-497a-a79d-73e648604286 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:02.835393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199e3e9-97ce-497a-a79d-73e648604286 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:02.840974 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 361/718] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 31 19:06:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:02.852043 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448eda2-6d2b-449b-b617-c12e9091b77e None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:02.943811 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:02.987774 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448eda2-6d2b-449b-b617-c12e9091b77e None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:03.026819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5474a8-aefa-40e1-b1c4-6c45b044f397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:03.027875 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5474a8-aefa-40e1-b1c4-6c45b044f397 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:03.028364 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5474a8-aefa-40e1-b1c4-6c45b044f397 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:03.028833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5474a8-aefa-40e1-b1c4-6c45b044f397 None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:03.029676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5474a8-aefa-40e1-b1c4-6c45b044f397 None admin] RBAC: Adding query filter params (type=network) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:03.030176 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5474a8-aefa-40e1-b1c4-6c45b044f397 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:03.031887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5474a8-aefa-40e1-b1c4-6c45b044f397 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:03.039464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 358/719] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 31 19:06:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:03.177844 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d26467-3abe-4cb4-9462-d28aa3735b40 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:03.178220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d26467-3abe-4cb4-9462-d28aa3735b40 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:03.178412 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d26467-3abe-4cb4-9462-d28aa3735b40 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:03.180055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 362/720] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:03.185582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd1c2d-f01d-46c8-8171-6bc54d3b5e19 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:03.185873 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd1c2d-f01d-46c8-8171-6bc54d3b5e19 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:03.186157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd1c2d-f01d-46c8-8171-6bc54d3b5e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:03.205433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-b5dd1c2d-f01d-46c8-8171-6bc54d3b5e19 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:03.207372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b5dd1c2d-f01d-46c8-8171-6bc54d3b5e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:03.313578 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 359/721] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:06:03.363691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fc230-208d-45d4-aedd-dff1bbd1c294 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:03.364308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fc230-208d-45d4-aedd-dff1bbd1c294 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:03.364791 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fc230-208d-45d4-aedd-dff1bbd1c294 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:03.366617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 363/722] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:03.372037 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b92a5-18a2-45a8-8474-316317b7187f None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:03.372334 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b92a5-18a2-45a8-8474-316317b7187f None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:03.372610 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b92a5-18a2-45a8-8474-316317b7187f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:03.398090 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-8f0b92a5-18a2-45a8-8474-316317b7187f None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:03.399322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8f0b92a5-18a2-45a8-8474-316317b7187f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:03.464912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:03.489920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 360/723] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:06:03.708305 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fc230-208d-45d4-aedd-dff1bbd1c294 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:03.820492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b92a5-18a2-45a8-8474-316317b7187f None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:03.863817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fc230-208d-45d4-aedd-dff1bbd1c294 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:03.881812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:03.928142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf982dfb-4008-452b-b977-fe550f9fa466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:03.929283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf982dfb-4008-452b-b977-fe550f9fa466 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:03.929856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf982dfb-4008-452b-b977-fe550f9fa466 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:03.930143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf982dfb-4008-452b-b977-fe550f9fa466 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:03.959219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf982dfb-4008-452b-b977-fe550f9fa466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:03.976131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-cf982dfb-4008-452b-b977-fe550f9fa466 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:03.976131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:03.976131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:03.976131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:03.976131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:03.976131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:03.976131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:03.976131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:03.976131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:03.976131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:03.976131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:03.976131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:03.976131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:03.976131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:03.976131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:03.976131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:03.977092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:03.977092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:03.977092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:03.977092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:03.977092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:03.977092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:03.977092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:03.977092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:03.977092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:03.977092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:03.977092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:03.977092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:03.977092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:03.977092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:03.977092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:03.977092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:03.977949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:03.977949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:03.977949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:03.977949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:03.977949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:03.977949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:03.977949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:03.977949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:03.977949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:03.977949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:03.977949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:03.977949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:03.979657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 364/724] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:06:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:03.985916 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf982dfb-4008-452b-b977-fe550f9fa466 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:04.028719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b92a5-18a2-45a8-8474-316317b7187f None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:04.124188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925c50b-f942-4814-811a-228583aa6d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:04.125170 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925c50b-f942-4814-811a-228583aa6d22 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:04.125650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925c50b-f942-4814-811a-228583aa6d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:04.126119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925c50b-f942-4814-811a-228583aa6d22 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:04.130894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf982dfb-4008-452b-b977-fe550f9fa466 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:04.151142 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925c50b-f942-4814-811a-228583aa6d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:04.161235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-9925c50b-f942-4814-811a-228583aa6d22 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:06:04.161235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:04.161235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:04.161235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:04.161235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:04.161235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:04.161235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:04.161235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:04.161235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:04.161235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:04.161235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:04.161235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:04.161235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:04.161235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:04.161235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:04.161235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:04.162311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:04.162311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:04.162311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:04.162311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:04.162311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:04.162311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:04.162311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:04.162311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:04.162311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:04.162311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:04.162311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:04.162311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:04.162311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:04.162311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:04.162311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:04.162311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:04.163187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:04.163187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:04.163187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:04.163187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:04.163187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:04.163187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:04.163187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:04.163187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:04.163187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:04.163187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:04.163187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:06:04.163187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:04.163187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 361/725] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:06:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:04.170782 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925c50b-f942-4814-811a-228583aa6d22 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:04.189805 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5d32f-8f2d-46d0-a715-6019fcbccff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:04.190525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5d32f-8f2d-46d0-a715-6019fcbccff3 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:04.191040 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5d32f-8f2d-46d0-a715-6019fcbccff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:04.191320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5d32f-8f2d-46d0-a715-6019fcbccff3 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:04.192008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5d32f-8f2d-46d0-a715-6019fcbccff3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:04.192316 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5d32f-8f2d-46d0-a715-6019fcbccff3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:04.198427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5d32f-8f2d-46d0-a715-6019fcbccff3 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:04.219388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 365/726] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:06:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:04.223765 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5d32f-8f2d-46d0-a715-6019fcbccff3 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:04.272518 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:04.295469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:04.395646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925c50b-f942-4814-811a-228583aa6d22 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:04.414556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5d32f-8f2d-46d0-a715-6019fcbccff3 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:04.467184 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569048e9-2df1-4812-8090-242ea843d270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:04.467803 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569048e9-2df1-4812-8090-242ea843d270 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:04.468084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569048e9-2df1-4812-8090-242ea843d270 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:04.468360 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569048e9-2df1-4812-8090-242ea843d270 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:04.469881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569048e9-2df1-4812-8090-242ea843d270 None admin] RBAC: Adding query filter params (name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:04.470202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569048e9-2df1-4812-8090-242ea843d270 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:04.472408 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569048e9-2df1-4812-8090-242ea843d270 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:04.490639 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 362/727] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:06:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:04.491313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49306a75-fd08-47f2-a0c6-36e021d55035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:04.491964 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49306a75-fd08-47f2-a0c6-36e021d55035 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:04.492245 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49306a75-fd08-47f2-a0c6-36e021d55035 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:04.492518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49306a75-fd08-47f2-a0c6-36e021d55035 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:04.495448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569048e9-2df1-4812-8090-242ea843d270 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:04.512690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49306a75-fd08-47f2-a0c6-36e021d55035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:04.514231 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49306a75-fd08-47f2-a0c6-36e021d55035 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:04.532515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-49306a75-fd08-47f2-a0c6-36e021d55035 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:04.532515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:04.532515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:04.532515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:04.532515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:04.532515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:04.532515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:04.532515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:04.532515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:04.532515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:04.532515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:04.532515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:04.532515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:04.532515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:04.532515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:04.532515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:04.533508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:04.533508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:04.533508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:04.533508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:04.533508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:04.533508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:04.533508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:04.533508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:04.533508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:04.533508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:04.533508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:04.533508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:04.533508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:04.533508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:04.533508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:04.533508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:04.533508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:04.535411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:04.535411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:04.535411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:04.535411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 366/728] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:04.540497 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49306a75-fd08-47f2-a0c6-36e021d55035 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:04.661010 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:04.670395 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569048e9-2df1-4812-8090-242ea843d270 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:04.680815 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:04.684175 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49306a75-fd08-47f2-a0c6-36e021d55035 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:04.714582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01876c0-e6b4-427d-81cb-221f215d3a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:04.715455 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01876c0-e6b4-427d-81cb-221f215d3a2a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:04.715798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01876c0-e6b4-427d-81cb-221f215d3a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:04.716137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01876c0-e6b4-427d-81cb-221f215d3a2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:04.729731 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01876c0-e6b4-427d-81cb-221f215d3a2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:04.729906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c20b9-dbf5-4de2-aacb-e4b3641e3240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:04.730460 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c20b9-dbf5-4de2-aacb-e4b3641e3240 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:04.730751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c20b9-dbf5-4de2-aacb-e4b3641e3240 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:04.731061 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c20b9-dbf5-4de2-aacb-e4b3641e3240 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:04.731556 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01876c0-e6b4-427d-81cb-221f215d3a2a None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:04.731712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c20b9-dbf5-4de2-aacb-e4b3641e3240 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:04.732083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c20b9-dbf5-4de2-aacb-e4b3641e3240 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:04.733849 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c20b9-dbf5-4de2-aacb-e4b3641e3240 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:04.747423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-a01876c0-e6b4-427d-81cb-221f215d3a2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:04.747423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:04.747423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:04.747423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:04.747423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:04.747423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:04.747423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:04.747423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:04.747423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:04.747423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:04.747423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:04.747423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:04.747423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:04.747423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:04.747423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:04.747423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:04.748417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:04.748417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:04.748417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:04.748417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:04.748417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:04.748417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:04.748417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:04.748417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:04.748417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:04.748417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:04.748417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:04.748417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:04.748417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:04.748417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:04.748417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:04.748417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:04.748417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:04.750369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:04.750369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:04.750369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:04.750762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 367/730] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:04.750762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e94b8-8e44-4a58-a448-b3c0976d3f91 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:04.750762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e94b8-8e44-4a58-a448-b3c0976d3f91 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:04.751240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e94b8-8e44-4a58-a448-b3c0976d3f91 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:04.752858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 368/731] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:04.754065 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e94b8-8e44-4a58-a448-b3c0976d3f91 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:04.758570 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 363/731] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:04.758570 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01876c0-e6b4-427d-81cb-221f215d3a2a None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:04.874670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e94b8-8e44-4a58-a448-b3c0976d3f91 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:04.906148 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01876c0-e6b4-427d-81cb-221f215d3a2a None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:04.914089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a566fd-619f-4b19-9ded-d76fa731dddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:04.914663 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a566fd-619f-4b19-9ded-d76fa731dddc None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:04.914968 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a566fd-619f-4b19-9ded-d76fa731dddc None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:04.915242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a566fd-619f-4b19-9ded-d76fa731dddc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:04.923276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a566fd-619f-4b19-9ded-d76fa731dddc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:04.925478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a566fd-619f-4b19-9ded-d76fa731dddc None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:04.938189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-e7a566fd-619f-4b19-9ded-d76fa731dddc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:06:04.938189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:04.938189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:04.938189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:04.938189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:04.938189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:04.938189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:04.938189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:04.938189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:04.938189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:04.938189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:04.938189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:04.938189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:06:04.938189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:06:04.938189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:06:04.938189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:04.941185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:04.945160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:04.945160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:04.945160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:06:04.945160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:06:04.945160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:06:04.945160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:06:04.945160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:06:04.945160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:06:04.945160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:06:04.945160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:04.945160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 369/732] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:06:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:04.945160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafa3bd-06ad-4afa-b4ba-f4099ec03865 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:04.945160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafa3bd-06ad-4afa-b4ba-f4099ec03865 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:04.946802 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafa3bd-06ad-4afa-b4ba-f4099ec03865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:04.950205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:04.952019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b3412-d644-4139-96e7-4d940bbdaad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:04.952818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b3412-d644-4139-96e7-4d940bbdaad1 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:04.953248 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b3412-d644-4139-96e7-4d940bbdaad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:04.953670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b3412-d644-4139-96e7-4d940bbdaad1 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:04.954525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b3412-d644-4139-96e7-4d940bbdaad1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:04.955060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b3412-d644-4139-96e7-4d940bbdaad1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:04.957246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b3412-d644-4139-96e7-4d940bbdaad1 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:04.962581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-9eafa3bd-06ad-4afa-b4ba-f4099ec03865 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:04.964079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9eafa3bd-06ad-4afa-b4ba-f4099ec03865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:04.973157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 364/733] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:04.974248 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b3412-d644-4139-96e7-4d940bbdaad1 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:05.008109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:05.055179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 370/734] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:06:05.056384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafa3bd-06ad-4afa-b4ba-f4099ec03865 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:05.107904 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b3412-d644-4139-96e7-4d940bbdaad1 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:05.153219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a8570-2516-414c-9893-12dd4de56f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:05.154081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a8570-2516-414c-9893-12dd4de56f0c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:05.154500 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a8570-2516-414c-9893-12dd4de56f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:05.154933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a8570-2516-414c-9893-12dd4de56f0c None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:05.155803 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a8570-2516-414c-9893-12dd4de56f0c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:05.156340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a8570-2516-414c-9893-12dd4de56f0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:05.158108 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a8570-2516-414c-9893-12dd4de56f0c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:05.180098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 365/735] 10.222.0.59 () {56 vars in 1135 bytes} [Wed Aug 31 19:06:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:05.187276 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a8570-2516-414c-9893-12dd4de56f0c None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:05.237845 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafa3bd-06ad-4afa-b4ba-f4099ec03865 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:05.277912 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a4dff-bacf-482c-ba85-6df4098107fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:05.278811 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a4dff-bacf-482c-ba85-6df4098107fc None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:05.279319 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a4dff-bacf-482c-ba85-6df4098107fc None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:05.279807 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a4dff-bacf-482c-ba85-6df4098107fc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:05.282542 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:05.290007 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a4dff-bacf-482c-ba85-6df4098107fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:05.291779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a4dff-bacf-482c-ba85-6df4098107fc None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:05.304894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-911a4dff-bacf-482c-ba85-6df4098107fc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:06:05.304894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:05.304894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:05.304894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:05.304894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:05.304894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:05.304894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:05.304894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:05.304894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:05.304894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:05.304894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:05.304894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:05.304894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:06:05.304894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:06:05.304894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:06:05.304894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:05.305908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:05.306889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:05.306889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:05.306889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:06:05.306889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:06:05.306889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:06:05.306889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:06:05.306889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:06:05.306889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:06:05.306889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:06:05.306889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:05.309414 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 371/736] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 31 19:06:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:05.317020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e3328-83c6-4fdf-bf37-aa060e2a61b1 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:05.317578 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:05.318281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e3328-83c6-4fdf-bf37-aa060e2a61b1 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:05.319263 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e3328-83c6-4fdf-bf37-aa060e2a61b1 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:05.321099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 372/737] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:05.322103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e3328-83c6-4fdf-bf37-aa060e2a61b1 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:05.346373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a8570-2516-414c-9893-12dd4de56f0c None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:05.386856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555369fc-0af1-4d91-8f5e-9c2af552bfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:05.387512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555369fc-0af1-4d91-8f5e-9c2af552bfaf None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:05.387794 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555369fc-0af1-4d91-8f5e-9c2af552bfaf None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:05.388068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555369fc-0af1-4d91-8f5e-9c2af552bfaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:05.438231 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e3328-83c6-4fdf-bf37-aa060e2a61b1 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:05.461588 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555369fc-0af1-4d91-8f5e-9c2af552bfaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:05.463068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555369fc-0af1-4d91-8f5e-9c2af552bfaf None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:05.472470 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4d1a4-03de-47df-bc14-88da5b1f5164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:05.473081 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4d1a4-03de-47df-bc14-88da5b1f5164 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:05.473359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4d1a4-03de-47df-bc14-88da5b1f5164 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:05.473630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4d1a4-03de-47df-bc14-88da5b1f5164 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:05.474370 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4d1a4-03de-47df-bc14-88da5b1f5164 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:05.474655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4d1a4-03de-47df-bc14-88da5b1f5164 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:05.476114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4d1a4-03de-47df-bc14-88da5b1f5164 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:05.494716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 373/738] 10.222.0.59 () {56 vars in 1149 bytes} [Wed Aug 31 19:06:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:05.495967 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563a1d1-c7da-4788-be67-4e5e46a0933e None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:05.496254 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563a1d1-c7da-4788-be67-4e5e46a0933e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:05.496518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563a1d1-c7da-4788-be67-4e5e46a0933e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:05.514976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-4563a1d1-c7da-4788-be67-4e5e46a0933e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:05.516178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4563a1d1-c7da-4788-be67-4e5e46a0933e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:05.544696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:05.565934 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:05.567192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 366/739] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:06:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 19:06:05.568527 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc14d61-710a-4389-a88d-67accb8b12f7 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:05.568815 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc14d61-710a-4389-a88d-67accb8b12f7 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:05.569089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc14d61-710a-4389-a88d-67accb8b12f7 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:05.570305 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 367/740] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:05.571053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc14d61-710a-4389-a88d-67accb8b12f7 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:05.591122 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 374/741] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:06:05.591973 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563a1d1-c7da-4788-be67-4e5e46a0933e None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:05.693663 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc14d61-710a-4389-a88d-67accb8b12f7 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:05.704985 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563a1d1-c7da-4788-be67-4e5e46a0933e None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:05.730602 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a250ed-967b-4c32-88d9-25bd9f09e0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:05.731299 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a250ed-967b-4c32-88d9-25bd9f09e0ac None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:05.731573 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a250ed-967b-4c32-88d9-25bd9f09e0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:05.731896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a250ed-967b-4c32-88d9-25bd9f09e0ac None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:05.732632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a250ed-967b-4c32-88d9-25bd9f09e0ac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:05.733978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a250ed-967b-4c32-88d9-25bd9f09e0ac None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:05.744013 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30348c-1ac1-48aa-b3fd-04b0a3f9e8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:05.744646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30348c-1ac1-48aa-b3fd-04b0a3f9e8bf None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:05.744911 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30348c-1ac1-48aa-b3fd-04b0a3f9e8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:05.745184 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30348c-1ac1-48aa-b3fd-04b0a3f9e8bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:05.798851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:05.817563 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30348c-1ac1-48aa-b3fd-04b0a3f9e8bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:05.819118 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30348c-1ac1-48aa-b3fd-04b0a3f9e8bf None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:05.831176 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 368/742] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 31 19:06:05 2022] POST /identity/v3/services => generated 251 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:05.831176 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a250ed-967b-4c32-88d9-25bd9f09e0ac None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:05.887132 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:05.939641 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 375/743] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:06:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:06:05.941898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57dcff8-db95-4893-8094-cb63d31c9562 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:05.942186 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57dcff8-db95-4893-8094-cb63d31c9562 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:05.942463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57dcff8-db95-4893-8094-cb63d31c9562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:05.962615 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d57dcff8-db95-4893-8094-cb63d31c9562 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:05.963854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d57dcff8-db95-4893-8094-cb63d31c9562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:05.990546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a250ed-967b-4c32-88d9-25bd9f09e0ac None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:06.031479 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffb2c8-f8b0-45b6-a5c5-32baa4efcf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:06.032390 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffb2c8-f8b0-45b6-a5c5-32baa4efcf4e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:06.032820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffb2c8-f8b0-45b6-a5c5-32baa4efcf4e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:06.033239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffb2c8-f8b0-45b6-a5c5-32baa4efcf4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:06.046895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 376/744] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:06:06.046895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57dcff8-db95-4893-8094-cb63d31c9562 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:06.051519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ffb2c8-f8b0-45b6-a5c5-32baa4efcf4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:06.053678 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ffb2c8-f8b0-45b6-a5c5-32baa4efcf4e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:06.059245 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:06.076632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-a2ffb2c8-f8b0-45b6-a5c5-32baa4efcf4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:06.076632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:06.076632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:06.076632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:06.076632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:06.076632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:06.076632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:06.076632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:06.076632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:06.076632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:06.076632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:06.076632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:06.076632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:06.076632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:06.076632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:06.076632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:06.077755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:06.077755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:06.077755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:06.077755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:06.077755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:06.077755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:06.077755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:06.077755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:06.077755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:06.077755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:06.077755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:06.077755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:06.077755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:06.077755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:06.077755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:06.077755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:06.077755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:06.078684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:06.078684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:06.078684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:06.079302 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 369/745] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:06.080410 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffb2c8-f8b0-45b6-a5c5-32baa4efcf4e None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:06.149142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57dcff8-db95-4893-8094-cb63d31c9562 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:06.188099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b48a0-5cfb-4f64-bd7a-c8c02c6d8c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:06.188811 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b48a0-5cfb-4f64-bd7a-c8c02c6d8c4f None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:06.189097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b48a0-5cfb-4f64-bd7a-c8c02c6d8c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:06.189375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b48a0-5cfb-4f64-bd7a-c8c02c6d8c4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:06.199012 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:06.201025 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b48a0-5cfb-4f64-bd7a-c8c02c6d8c4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:06.206684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-ee4b48a0-5cfb-4f64-bd7a-c8c02c6d8c4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:06.206684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:06.206684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:06.206684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:06.206684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:06.206684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:06.206684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:06.206684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:06.206684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:06.206684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:06.206684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:06.206684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:06.206684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:06.206684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:06.206684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:06.206684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:06.208101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:06.208101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:06.208101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:06.208101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:06.208101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:06.208101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:06.208101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:06.208101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:06.208101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:06.208101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:06.208101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:06.208101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:06.208101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:06.208101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:06.208101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:06.208101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:06.209304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:06.209304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:06.209304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:06.209304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:06.209304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:06.209304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:06.209304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:06.209304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:06.209304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:06.209304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:06.209304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:06.209304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:06.209304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 377/746] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:06:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:06.210896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b48a0-5cfb-4f64-bd7a-c8c02c6d8c4f None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:06.224598 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffb2c8-f8b0-45b6-a5c5-32baa4efcf4e None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:06.280958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acc955-63e8-4eb3-ab4a-4c4ab0336fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:06.282123 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acc955-63e8-4eb3-ab4a-4c4ab0336fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:06.282964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acc955-63e8-4eb3-ab4a-4c4ab0336fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:06.283597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acc955-63e8-4eb3-ab4a-4c4ab0336fa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:06.298255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:06.305126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5acc955-63e8-4eb3-ab4a-4c4ab0336fa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:06.307377 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5acc955-63e8-4eb3-ab4a-4c4ab0336fa0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:06.328281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-b5acc955-63e8-4eb3-ab4a-4c4ab0336fa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:06.328281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:06.328281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:06.328281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:06.328281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:06.328281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:06.328281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:06.328281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:06.328281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:06.328281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:06.328281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:06.328281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:06.328281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:06.328281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:06.328281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:06.328281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:06.329382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:06.329382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:06.329382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:06.329382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:06.329382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:06.329382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:06.329382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:06.329382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:06.329382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:06.329382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:06.329382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:06.329382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:06.329382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:06.329382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:06.329382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:06.329382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:06.329382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:06.330287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:06.330287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:06.330287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:06.337634 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b48a0-5cfb-4f64-bd7a-c8c02c6d8c4f None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:06.338133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 370/747] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:06.341103 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acc955-63e8-4eb3-ab4a-4c4ab0336fa0 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:06.379484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009a2cd-3f52-447d-b098-deb422f4ef3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:06.380208 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009a2cd-3f52-447d-b098-deb422f4ef3a None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:06.380491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009a2cd-3f52-447d-b098-deb422f4ef3a None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:06.380768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009a2cd-3f52-447d-b098-deb422f4ef3a None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:06.381422 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009a2cd-3f52-447d-b098-deb422f4ef3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:06.381727 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009a2cd-3f52-447d-b098-deb422f4ef3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:06.383128 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009a2cd-3f52-447d-b098-deb422f4ef3a None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:06.398166 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 378/748] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:06.398166 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009a2cd-3f52-447d-b098-deb422f4ef3a None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:06.452237 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acc955-63e8-4eb3-ab4a-4c4ab0336fa0 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:06.490491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44b8db-3d25-4f69-b04a-144287c838ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:06.491055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44b8db-3d25-4f69-b04a-144287c838ef None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:06.491340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44b8db-3d25-4f69-b04a-144287c838ef None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:06.491616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44b8db-3d25-4f69-b04a-144287c838ef None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:06.492279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44b8db-3d25-4f69-b04a-144287c838ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:06.492584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44b8db-3d25-4f69-b04a-144287c838ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:06.493968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44b8db-3d25-4f69-b04a-144287c838ef None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:06.499365 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:06.504848 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 371/749] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:06:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:06.505636 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44b8db-3d25-4f69-b04a-144287c838ef None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:06.520291 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009a2cd-3f52-447d-b098-deb422f4ef3a None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:06.551237 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:06.557410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489efa05-9620-429d-bdaf-7c5e544cfa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:06.558021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489efa05-9620-429d-bdaf-7c5e544cfa89 None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:06.558298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489efa05-9620-429d-bdaf-7c5e544cfa89 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:06.558569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489efa05-9620-429d-bdaf-7c5e544cfa89 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:06.559255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489efa05-9620-429d-bdaf-7c5e544cfa89 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:06.560620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489efa05-9620-429d-bdaf-7c5e544cfa89 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:06.612551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44b8db-3d25-4f69-b04a-144287c838ef None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:06.625560 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 379/750] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 31 19:06:06 2022] POST /identity/v3/services => generated 240 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:06.626827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489efa05-9620-429d-bdaf-7c5e544cfa89 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:06.669862 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783f683-25fc-4117-bda2-e5dfff4ba2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:06.670613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783f683-25fc-4117-bda2-e5dfff4ba2ac None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:06.670975 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783f683-25fc-4117-bda2-e5dfff4ba2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:06.671408 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783f683-25fc-4117-bda2-e5dfff4ba2ac None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:06.672516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783f683-25fc-4117-bda2-e5dfff4ba2ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:06.672822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783f683-25fc-4117-bda2-e5dfff4ba2ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:06.675106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783f683-25fc-4117-bda2-e5dfff4ba2ac None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:06.692964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 372/751] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:06.694400 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783f683-25fc-4117-bda2-e5dfff4ba2ac None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:06.764043 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:06.770490 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489efa05-9620-429d-bdaf-7c5e544cfa89 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:06.818774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454c96c-20a3-4cf2-84f1-16188997b1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:06.820086 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454c96c-20a3-4cf2-84f1-16188997b1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:06.820995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454c96c-20a3-4cf2-84f1-16188997b1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:06.821380 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454c96c-20a3-4cf2-84f1-16188997b1d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:06.823741 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783f683-25fc-4117-bda2-e5dfff4ba2ac None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:06.830028 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454c96c-20a3-4cf2-84f1-16188997b1d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:06.830595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454c96c-20a3-4cf2-84f1-16188997b1d8 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:06.842651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-9454c96c-20a3-4cf2-84f1-16188997b1d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:06.842651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:06.842651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:06.842651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:06.842651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:06.842651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:06.842651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:06.842651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:06.842651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:06.842651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:06.842651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:06.842651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:06.842651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:06:06.842651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:06:06.842651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:06:06.844104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:06:06.844104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:06.844104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:06.844104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:06.844104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:06.844104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:06.844104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:06.844104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:06.844104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:06.844104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:06.844104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:06.844104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:06.844104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:06.844104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:06.844104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:06.844104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:06.844104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:06.845046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:06:06.845046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:06:06.845046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:06:06.845046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:06:06.845046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:06:06.845046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:06:06.845046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:06.845046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:06.845046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 380/752] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:06.845046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454c96c-20a3-4cf2-84f1-16188997b1d8 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:06.868388 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a268e-de6a-4ce3-b76e-5f367962b46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:06.869066 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a268e-de6a-4ce3-b76e-5f367962b46f None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:06.869312 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a268e-de6a-4ce3-b76e-5f367962b46f None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:06.869584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a268e-de6a-4ce3-b76e-5f367962b46f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:06.881437 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a268e-de6a-4ce3-b76e-5f367962b46f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:06.882899 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a268e-de6a-4ce3-b76e-5f367962b46f None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:06.899304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-a95a268e-de6a-4ce3-b76e-5f367962b46f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:06.899304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:06.899304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:06.899304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:06.899304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:06.899304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:06.899304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:06.899304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:06.899304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:06.899304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:06.899304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:06.899304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:06.899304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:06.899304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:06.899304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:06.899304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:06.900326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:06.900326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:06.900326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:06.900326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:06.900326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:06.900326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:06.900326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:06.900326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:06.900326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:06.900326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:06.900326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:06.900326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:06.900326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:06.900326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:06.900326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:06.900326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:06.900326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:06.901244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:06.901244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:06.901244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:06.901244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 373/753] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:06.901898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a268e-de6a-4ce3-b76e-5f367962b46f None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:06.903496 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:06.961239 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454c96c-20a3-4cf2-84f1-16188997b1d8 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:07.009519 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc878c5-2a71-4e67-814f-19243d69a895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:07.010206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc878c5-2a71-4e67-814f-19243d69a895 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:07.010496 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc878c5-2a71-4e67-814f-19243d69a895 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:07.010773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc878c5-2a71-4e67-814f-19243d69a895 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:07.021022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc878c5-2a71-4e67-814f-19243d69a895 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:07.023073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc878c5-2a71-4e67-814f-19243d69a895 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:07.026383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a268e-de6a-4ce3-b76e-5f367962b46f None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:07.037806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-6cc878c5-2a71-4e67-814f-19243d69a895 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:07.037806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:07.037806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:07.037806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:07.037806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:07.037806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:07.037806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:07.037806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:07.037806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:07.037806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:07.037806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:07.037806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:07.037806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:06:07.037806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:06:07.037806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:06:07.038784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:06:07.038784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:07.038784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:07.038784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:07.038784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:07.038784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:07.038784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:07.038784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:07.038784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:07.038784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:07.038784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:07.038784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:07.038784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:07.038784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:07.038784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:07.038784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:07.038784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:07.039975 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:06:07.039975 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:06:07.039975 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:06:07.039975 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:06:07.039975 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:06:07.039975 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:06:07.039975 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:07.039975 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:07.039975 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 381/754] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:07.040598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc878c5-2a71-4e67-814f-19243d69a895 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:07.044054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:07.068591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47f47b-f693-48ed-a3b6-eea090e548e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:07.069208 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47f47b-f693-48ed-a3b6-eea090e548e3 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:07.069490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47f47b-f693-48ed-a3b6-eea090e548e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:07.070741 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47f47b-f693-48ed-a3b6-eea090e548e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:07.071489 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47f47b-f693-48ed-a3b6-eea090e548e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:07.071970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47f47b-f693-48ed-a3b6-eea090e548e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:07.073523 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47f47b-f693-48ed-a3b6-eea090e548e3 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:07.087831 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 374/755] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:06:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:07.088850 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47f47b-f693-48ed-a3b6-eea090e548e3 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:07.175511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc878c5-2a71-4e67-814f-19243d69a895 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:07.188798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:07.211706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e471b91-b781-4cc9-8bfd-2e26058f377e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:07.212441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47f47b-f693-48ed-a3b6-eea090e548e3 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:07.212955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e471b91-b781-4cc9-8bfd-2e26058f377e None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:07.213433 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e471b91-b781-4cc9-8bfd-2e26058f377e None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:07.213928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e471b91-b781-4cc9-8bfd-2e26058f377e None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:07.214773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e471b91-b781-4cc9-8bfd-2e26058f377e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:07.215322 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e471b91-b781-4cc9-8bfd-2e26058f377e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:07.216979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e471b91-b781-4cc9-8bfd-2e26058f377e None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:07.231784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 382/756] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:07.233007 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e471b91-b781-4cc9-8bfd-2e26058f377e None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:07.258219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b0724-279f-4d04-9822-34446b69a662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:07.258835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b0724-279f-4d04-9822-34446b69a662 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:07.259177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b0724-279f-4d04-9822-34446b69a662 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:07.259886 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b0724-279f-4d04-9822-34446b69a662 None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:07.260596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b0724-279f-4d04-9822-34446b69a662 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:07.260973 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b0724-279f-4d04-9822-34446b69a662 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:07.262842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b0724-279f-4d04-9822-34446b69a662 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:07.282544 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 375/757] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:06:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:07.282544 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b0724-279f-4d04-9822-34446b69a662 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:07.324704 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:07.364650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e471b91-b781-4cc9-8bfd-2e26058f377e None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:07.404122 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c5d6c-5932-4862-9823-d31319be8c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:07.405081 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c5d6c-5932-4862-9823-d31319be8c88 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:07.405590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c5d6c-5932-4862-9823-d31319be8c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:07.406167 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c5d6c-5932-4862-9823-d31319be8c88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:07.407238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c5d6c-5932-4862-9823-d31319be8c88 None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=25ed58369ffd46e8b7fa1880662926e8) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:07.407749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c5d6c-5932-4862-9823-d31319be8c88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:07.410125 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c5d6c-5932-4862-9823-d31319be8c88 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:07.410543 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b0724-279f-4d04-9822-34446b69a662 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:07.417183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 383/758] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:06:07 2022] GET /identity/v3/role_assignments?user.id=25ed58369ffd46e8b7fa1880662926e8&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:07.418463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c5d6c-5932-4862-9823-d31319be8c88 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:07.448121 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6dc7e-dbb1-44d6-95de-ac40bc53dd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:07.448774 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6dc7e-dbb1-44d6-95de-ac40bc53dd59 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:07.449056 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6dc7e-dbb1-44d6-95de-ac40bc53dd59 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:07.449330 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6dc7e-dbb1-44d6-95de-ac40bc53dd59 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:07.458390 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6dc7e-dbb1-44d6-95de-ac40bc53dd59 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:07.459881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6dc7e-dbb1-44d6-95de-ac40bc53dd59 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:07.464238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:07.473092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-92c6dc7e-dbb1-44d6-95de-ac40bc53dd59 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:06:07.473092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:07.473092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:07.473092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:07.473092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:07.473092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:07.473092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:07.473092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:07.473092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:07.473092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:07.473092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:07.473092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:07.473092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:06:07.473092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:06:07.473092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:06:07.473092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:07.474110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:07.475506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:07.475506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:07.475506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:06:07.475506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:06:07.475506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:06:07.475506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:06:07.475506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:06:07.475506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:06:07.475506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:06:07.475506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:07.475506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 376/759] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 31 19:06:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:07.480528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6dc7e-dbb1-44d6-95de-ac40bc53dd59 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:07.546656 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c5d6c-5932-4862-9823-d31319be8c88 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:07.584906 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:07.589457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e09d8-930e-450d-8b42-99cccf2f9e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:07.590312 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e09d8-930e-450d-8b42-99cccf2f9e30 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:07.590789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e09d8-930e-450d-8b42-99cccf2f9e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:07.591292 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e09d8-930e-450d-8b42-99cccf2f9e30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:07.592348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e09d8-930e-450d-8b42-99cccf2f9e30 None admin] RBAC: Adding query filter params (role.id=a9ef0098e982437fb0127827fa2cad38, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=1ee990b1658e49cc8bc2fe15e7dc36d9) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:07.597052 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e09d8-930e-450d-8b42-99cccf2f9e30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:07.597052 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e09d8-930e-450d-8b42-99cccf2f9e30 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:07.598513 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6dc7e-dbb1-44d6-95de-ac40bc53dd59 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:07.607037 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 384/760] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:06:07 2022] GET /identity/v3/role_assignments?user.id=1ee990b1658e49cc8bc2fe15e7dc36d9&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=a9ef0098e982437fb0127827fa2cad38 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:07.639940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd05d93-e1ae-46bf-b4b4-e0ab43c2b11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:07.640588 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd05d93-e1ae-46bf-b4b4-e0ab43c2b11e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:07.640869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd05d93-e1ae-46bf-b4b4-e0ab43c2b11e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:07.641137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd05d93-e1ae-46bf-b4b4-e0ab43c2b11e None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:07.641906 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd05d93-e1ae-46bf-b4b4-e0ab43c2b11e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:07.642213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd05d93-e1ae-46bf-b4b4-e0ab43c2b11e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:07.644496 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd05d93-e1ae-46bf-b4b4-e0ab43c2b11e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:07.667053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 377/761] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 31 19:06:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:07.672397 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e09d8-930e-450d-8b42-99cccf2f9e30 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:07.828895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:07.833387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e09d8-930e-450d-8b42-99cccf2f9e30 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:07.878451 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09e1e5-c4c6-4c07-bb83-a9d93248c7fb None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:07.878762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09e1e5-c4c6-4c07-bb83-a9d93248c7fb None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:07.879029 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09e1e5-c4c6-4c07-bb83-a9d93248c7fb None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:07.880600 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 378/762] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:07.881085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7144f5-05a3-4537-9df0-cbc91fcefa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:07.881751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7144f5-05a3-4537-9df0-cbc91fcefa35 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:07.882038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7144f5-05a3-4537-9df0-cbc91fcefa35 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:07.882314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7144f5-05a3-4537-9df0-cbc91fcefa35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:07.885527 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773f6e6-50d5-4c2a-8a20-f22e20c1cb14 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:07.885811 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773f6e6-50d5-4c2a-8a20-f22e20c1cb14 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:07.886089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773f6e6-50d5-4c2a-8a20-f22e20c1cb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:07.898704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7144f5-05a3-4537-9df0-cbc91fcefa35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:07.900220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7144f5-05a3-4537-9df0-cbc91fcefa35 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:07.905077 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-6773f6e6-50d5-4c2a-8a20-f22e20c1cb14 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:07.906598 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-6773f6e6-50d5-4c2a-8a20-f22e20c1cb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:07.925264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-2c7144f5-05a3-4537-9df0-cbc91fcefa35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:07.925264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:07.925264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:07.925264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:07.925264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:07.925264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:07.925264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:07.925264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:07.925264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:07.925264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:07.925264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:07.925264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:07.925264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:07.925264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:07.925264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:07.925264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:07.926342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:07.926342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:07.926342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:07.926342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:07.926342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:07.926342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:07.926342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:07.926342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:07.926342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:07.926342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:07.926342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:07.926342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:07.926342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:07.926342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:07.926342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:07.926342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:07.926342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:07.927290 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:07.927290 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:07.927290 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:07.929420 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 385/763] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:07.935080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7144f5-05a3-4537-9df0-cbc91fcefa35 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:08.057412 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7144f5-05a3-4537-9df0-cbc91fcefa35 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:08.068599 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 379/764] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:08.118242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60530f-08e1-4119-b68e-538aa5f68ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:08.118779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60530f-08e1-4119-b68e-538aa5f68ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:08.119102 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60530f-08e1-4119-b68e-538aa5f68ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:08.119374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60530f-08e1-4119-b68e-538aa5f68ff3 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:08.120026 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60530f-08e1-4119-b68e-538aa5f68ff3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:08.120328 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60530f-08e1-4119-b68e-538aa5f68ff3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:08.121692 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60530f-08e1-4119-b68e-538aa5f68ff3 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:08.134949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 386/765] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:08.140241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773f6e6-50d5-4c2a-8a20-f22e20c1cb14 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:08.237627 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:08.264240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773f6e6-50d5-4c2a-8a20-f22e20c1cb14 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:08.317991 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6169f-0c56-4395-a32e-5cbd55df387b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:08.319041 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6169f-0c56-4395-a32e-5cbd55df387b None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:08.319551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6169f-0c56-4395-a32e-5cbd55df387b None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:08.320058 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6169f-0c56-4395-a32e-5cbd55df387b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:08.331974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6169f-0c56-4395-a32e-5cbd55df387b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:08.333767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6169f-0c56-4395-a32e-5cbd55df387b None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:08.346204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-6fd6169f-0c56-4395-a32e-5cbd55df387b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:08.346204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:08.346204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:08.346204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:08.346204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:08.346204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:08.346204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:08.346204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:08.346204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:08.346204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:08.346204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:08.346204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:08.346204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:06:08.346204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:06:08.346204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:06:08.347401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:06:08.347401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:08.347401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:08.347401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:08.347401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:08.347401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:08.347401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:08.347401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:08.347401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:08.347401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:08.347401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:08.347401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:08.347401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:08.347401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:08.347401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:08.347401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:08.347401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:08.348402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:06:08.348402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:06:08.348402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:06:08.348402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:06:08.348402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:06:08.348402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:06:08.348402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:08.348402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:08.351034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 380/766] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:08.356319 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60530f-08e1-4119-b68e-538aa5f68ff3 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:08.379595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:08.467501 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60530f-08e1-4119-b68e-538aa5f68ff3 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:08.487448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6169f-0c56-4395-a32e-5cbd55df387b None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:08.508846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733b107-867d-49cc-ad23-75d81da5fbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:08.510285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733b107-867d-49cc-ad23-75d81da5fbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:08.510766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733b107-867d-49cc-ad23-75d81da5fbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:08.511534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733b107-867d-49cc-ad23-75d81da5fbb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:08.512433 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733b107-867d-49cc-ad23-75d81da5fbb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:08.512937 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733b107-867d-49cc-ad23-75d81da5fbb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:08.514564 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733b107-867d-49cc-ad23-75d81da5fbb1 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:08.528387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 387/767] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:06:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:08.536236 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733b107-867d-49cc-ad23-75d81da5fbb1 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:08.627276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:08.657328 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733b107-867d-49cc-ad23-75d81da5fbb1 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:08.694842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cb357-17e9-4f34-bdbe-4ad3d00c5b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:08.696072 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cb357-17e9-4f34-bdbe-4ad3d00c5b81 None admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:08.696551 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cb357-17e9-4f34-bdbe-4ad3d00c5b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:08.697053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cb357-17e9-4f34-bdbe-4ad3d00c5b81 None admin] PATH_INFO: `/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/d461422ce98442bcbe6b254a8dad5e8a/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:08.699187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cb357-17e9-4f34-bdbe-4ad3d00c5b81 None admin] RBAC: Authorizing `identity:create_grant(project_id=161c90d34bb34925a1aaeb70c7de7eae, user_id=d461422ce98442bcbe6b254a8dad5e8a, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:08.701131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cb357-17e9-4f34-bdbe-4ad3d00c5b81 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:08.741956 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6169f-0c56-4395-a32e-5cbd55df387b None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:08.778941 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de472b-099b-4a67-94e3-17ca17532345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:08.779588 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de472b-099b-4a67-94e3-17ca17532345 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:08.779867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de472b-099b-4a67-94e3-17ca17532345 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:08.780217 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de472b-099b-4a67-94e3-17ca17532345 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:08.780812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de472b-099b-4a67-94e3-17ca17532345 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:08.782173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de472b-099b-4a67-94e3-17ca17532345 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:08.786935 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:08.796807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-c7de472b-099b-4a67-94e3-17ca17532345 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:06:08.796807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:08.796807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:08.796807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:08.796807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:08.796807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:08.796807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:08.796807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:08.796807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:06:08.796807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:08.796807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:08.796807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:08.796807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:08.796807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:08.796807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:08.796807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:08.796807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:08.798198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:08.798198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:08.798198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:08.798198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:08.798198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:08.798198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:08.798198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:08.798198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:08.798198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:08.798198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:08.798198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:08.798198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:08.798198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:08.798198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:08.798198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:08.798198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:08.799843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:08.799843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:08.799843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:08.799843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:08.799843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:08.799843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:08.799843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:08.799843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:08.799843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:08.799843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:08.799843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:08.799843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:08.799843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:06:08.799843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:08.799843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 381/769] 10.222.0.59 () {56 vars in 1129 bytes} [Wed Aug 31 19:06:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:08.804224 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de472b-099b-4a67-94e3-17ca17532345 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:08.804411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 388/769] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:06:08 2022] PUT /identity/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/d461422ce98442bcbe6b254a8dad5e8a/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:06:08.921209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de472b-099b-4a67-94e3-17ca17532345 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:08.958900 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d386c52-02f6-4435-8d9d-5b516b0e9cc1 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:08.959621 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d386c52-02f6-4435-8d9d-5b516b0e9cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:08.960062 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc6d2a-cfaa-4a4b-a800-e52c473acc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:08.960365 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d386c52-02f6-4435-8d9d-5b516b0e9cc1 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:08.960678 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc6d2a-cfaa-4a4b-a800-e52c473acc17 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:08.960961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc6d2a-cfaa-4a4b-a800-e52c473acc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:08.961237 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc6d2a-cfaa-4a4b-a800-e52c473acc17 None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:08.961876 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc6d2a-cfaa-4a4b-a800-e52c473acc17 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:08.962184 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc6d2a-cfaa-4a4b-a800-e52c473acc17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:08.966759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc6d2a-cfaa-4a4b-a800-e52c473acc17 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:08.967080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 389/770] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:08.968901 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b029d3f-e30e-4ce5-b1c3-23995547bed4 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:08.969133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 382/771] 10.222.0.59 () {56 vars in 1124 bytes} [Wed Aug 31 19:06:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:08.969593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b029d3f-e30e-4ce5-b1c3-23995547bed4 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:08.969952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b029d3f-e30e-4ce5-b1c3-23995547bed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:08.975949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc6d2a-cfaa-4a4b-a800-e52c473acc17 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:08.991466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-5b029d3f-e30e-4ce5-b1c3-23995547bed4 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:08.992805 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-5b029d3f-e30e-4ce5-b1c3-23995547bed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:09.060097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:09.069903 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 390/772] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:09.103040 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc6d2a-cfaa-4a4b-a800-e52c473acc17 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:09.149312 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b540e-b878-47af-886d-faf593643fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:09.149931 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b540e-b878-47af-886d-faf593643fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:09.150213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b540e-b878-47af-886d-faf593643fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:09.150486 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b540e-b878-47af-886d-faf593643fd0 None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:09.151141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b540e-b878-47af-886d-faf593643fd0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:09.151447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b540e-b878-47af-886d-faf593643fd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:09.152874 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b540e-b878-47af-886d-faf593643fd0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:09.158601 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 383/773] 10.222.0.59 () {56 vars in 1124 bytes} [Wed Aug 31 19:06:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:09.163944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b029d3f-e30e-4ce5-b1c3-23995547bed4 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:09.231163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:09.274198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b029d3f-e30e-4ce5-b1c3-23995547bed4 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:09.311053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba2cdb-2747-451e-a248-1b1b1e6c93df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:09.311770 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba2cdb-2747-451e-a248-1b1b1e6c93df None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:09.312124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba2cdb-2747-451e-a248-1b1b1e6c93df None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:09.312469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba2cdb-2747-451e-a248-1b1b1e6c93df None admin] PATH_INFO: `/v3/endpoints` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:09.313258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba2cdb-2747-451e-a248-1b1b1e6c93df None admin] RBAC: Adding query filter params (interface=public, service_id=11bf9e6895bf4dbd9a490547d4178825, region_id=RegionOne) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:09.313642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba2cdb-2747-451e-a248-1b1b1e6c93df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:09.316063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba2cdb-2747-451e-a248-1b1b1e6c93df None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:09.324774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 391/774] 10.222.0.59 () {56 vars in 1251 bytes} [Wed Aug 31 19:06:09 2022] GET /identity/v3/endpoints?service_id=11bf9e6895bf4dbd9a490547d4178825&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:09.330461 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b540e-b878-47af-886d-faf593643fd0 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:09.349501 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba2cdb-2747-451e-a248-1b1b1e6c93df None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:09.444412 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b540e-b878-47af-886d-faf593643fd0 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:09.475481 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:09.482560 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77dc23-bff6-4233-9a31-d762c0ccbe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:09.483465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77dc23-bff6-4233-9a31-d762c0ccbe81 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:09.486813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77dc23-bff6-4233-9a31-d762c0ccbe81 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:09.486813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77dc23-bff6-4233-9a31-d762c0ccbe81 None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:09.486813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77dc23-bff6-4233-9a31-d762c0ccbe81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:09.486813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77dc23-bff6-4233-9a31-d762c0ccbe81 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:09.493490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 384/775] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 31 19:06:09 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:09.535354 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba2cdb-2747-451e-a248-1b1b1e6c93df None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:09.585746 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:09.602724 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59e29f-0711-49fe-9c89-caec77268db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:09.603418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59e29f-0711-49fe-9c89-caec77268db9 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:09.603696 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59e29f-0711-49fe-9c89-caec77268db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:09.603966 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59e29f-0711-49fe-9c89-caec77268db9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:09.604558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59e29f-0711-49fe-9c89-caec77268db9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:09.605936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59e29f-0711-49fe-9c89-caec77268db9 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:09.623926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-9f59e29f-0711-49fe-9c89-caec77268db9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:06:09.623926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:09.623926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:09.623926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:09.623926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:09.623926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:09.623926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:09.623926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:09.623926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:06:09.623926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:09.623926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:09.623926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:09.623926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:09.623926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:09.623926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:09.623926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:09.623926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:09.625015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:09.625015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:09.625015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:09.625015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:09.625015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:09.625015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:09.625015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:09.625015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:09.625015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:09.625015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:09.625015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:09.625015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:09.625015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:09.625015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:09.625015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:09.625015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:09.625882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:09.625882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:09.625882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:09.625882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:09.625882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:09.625882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:09.625882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:09.625882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:09.625882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:09.625882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:09.625882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:09.625882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:09.625882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:06:09.625882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:09.625882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 392/776] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:09 2022] GET /identity/v3/services/network => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:09.631618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77dc23-bff6-4233-9a31-d762c0ccbe81 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:09.754680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77dc23-bff6-4233-9a31-d762c0ccbe81 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:09.791789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c40207-f6e7-4f24-b7cf-ac7f5e50951e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:09.792793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c40207-f6e7-4f24-b7cf-ac7f5e50951e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:09.793276 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c40207-f6e7-4f24-b7cf-ac7f5e50951e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:09.793749 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c40207-f6e7-4f24-b7cf-ac7f5e50951e None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:09.794581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c40207-f6e7-4f24-b7cf-ac7f5e50951e None admin] RBAC: Adding query filter params (name=network) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:09.795141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c40207-f6e7-4f24-b7cf-ac7f5e50951e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:09.796722 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c40207-f6e7-4f24-b7cf-ac7f5e50951e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:09.802784 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 385/777] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 31 19:06:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:09.810353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59e29f-0711-49fe-9c89-caec77268db9 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:09.898028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ddbf0-ad80-4d81-b325-38be308e7867 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:09.898688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ddbf0-ad80-4d81-b325-38be308e7867 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:09.899234 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ddbf0-ad80-4d81-b325-38be308e7867 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:09.901006 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 386/778] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:09.906389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798ca4c-58cd-4c06-8e22-c738a01d3725 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:09.906966 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798ca4c-58cd-4c06-8e22-c738a01d3725 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:09.907443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798ca4c-58cd-4c06-8e22-c738a01d3725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:09.928516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-d798ca4c-58cd-4c06-8e22-c738a01d3725 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:09.930052 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d798ca4c-58cd-4c06-8e22-c738a01d3725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:09.939116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59e29f-0711-49fe-9c89-caec77268db9 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:10.015459 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:10.016980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e7da3-128b-4e6f-ae32-09dcd99a6532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:10.017706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e7da3-128b-4e6f-ae32-09dcd99a6532 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:10.018710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e7da3-128b-4e6f-ae32-09dcd99a6532 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:10.019111 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e7da3-128b-4e6f-ae32-09dcd99a6532 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:10.019856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e7da3-128b-4e6f-ae32-09dcd99a6532 None admin] RBAC: Adding query filter params (type=network) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:10.020722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e7da3-128b-4e6f-ae32-09dcd99a6532 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:10.022846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e7da3-128b-4e6f-ae32-09dcd99a6532 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:10.038266 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 393/779] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 31 19:06:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:10.041576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e7da3-128b-4e6f-ae32-09dcd99a6532 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:10.047537 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 387/780] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:10.147245 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:10.201043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e7da3-128b-4e6f-ae32-09dcd99a6532 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:10.222165 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da862dfc-43da-4aa0-89cc-74714fe7a7d5 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:10.222714 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da862dfc-43da-4aa0-89cc-74714fe7a7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:10.222714 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da862dfc-43da-4aa0-89cc-74714fe7a7d5 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:10.224378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 388/781] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:10.229353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9629f-bb34-4ab6-9e36-6774bafbd8d0 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:10.229643 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9629f-bb34-4ab6-9e36-6774bafbd8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:10.229923 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9629f-bb34-4ab6-9e36-6774bafbd8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:10.254926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-9ba9629f-bb34-4ab6-9e36-6774bafbd8d0 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:10.254926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-9ba9629f-bb34-4ab6-9e36-6774bafbd8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:10.275939 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1445e-71f4-48f6-ba51-f407f33c629a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:10.275939 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1445e-71f4-48f6-ba51-f407f33c629a None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:10.275939 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1445e-71f4-48f6-ba51-f407f33c629a None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:10.275939 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1445e-71f4-48f6-ba51-f407f33c629a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:10.276851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1445e-71f4-48f6-ba51-f407f33c629a None admin] RBAC: Adding query filter params (interface=public, service_id=ecf8a8f10dd246689cb9f08db3d6c310, region_id=RegionOne) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:10.276851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1445e-71f4-48f6-ba51-f407f33c629a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:10.276851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1445e-71f4-48f6-ba51-f407f33c629a None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:10.293049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 394/782] 10.222.0.59 () {56 vars in 1251 bytes} [Wed Aug 31 19:06:10 2022] GET /identity/v3/endpoints?service_id=ecf8a8f10dd246689cb9f08db3d6c310&interface=public®ion_id=RegionOne => generated 194 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:10.303941 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1445e-71f4-48f6-ba51-f407f33c629a None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:10.349975 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 389/783] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:10.355066 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9629f-bb34-4ab6-9e36-6774bafbd8d0 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:10.479105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9629f-bb34-4ab6-9e36-6774bafbd8d0 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:10.497925 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1445e-71f4-48f6-ba51-f407f33c629a None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:10.498929 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:10.521751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0ed09-2f70-49b9-9b9d-1d08a7753290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:10.522548 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0ed09-2f70-49b9-9b9d-1d08a7753290 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:10.522973 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0ed09-2f70-49b9-9b9d-1d08a7753290 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:10.523358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0ed09-2f70-49b9-9b9d-1d08a7753290 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:10.535561 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0ed09-2f70-49b9-9b9d-1d08a7753290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:10.540634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-a6f0ed09-2f70-49b9-9b9d-1d08a7753290 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:10.540634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:10.540634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:10.540634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:10.540634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:10.540634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:10.540634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:10.540634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:10.540634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:10.540634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:10.540634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:10.540634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:10.540634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:10.540634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:10.540634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:10.540634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:10.541769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:10.541769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:10.541769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:10.541769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:10.541769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:10.541769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:10.541769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:10.541769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:10.541769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:10.541769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:10.541769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:10.541769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:10.541769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:10.541769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:10.541769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:10.541769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:10.542852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:10.542852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:10.542852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:10.542852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:10.542852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:10.542852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:10.542852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:10.542852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:10.542852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:10.542852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:10.542852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:10.542852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:10.544790 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 390/784] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:06:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:10.546008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:10.550819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0ed09-2f70-49b9-9b9d-1d08a7753290 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:10.628103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1260d81-7110-4ec3-9271-d8d42ed00d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:10.628731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1260d81-7110-4ec3-9271-d8d42ed00d0d None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:10.629011 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1260d81-7110-4ec3-9271-d8d42ed00d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:10.629283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1260d81-7110-4ec3-9271-d8d42ed00d0d None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:10.629895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1260d81-7110-4ec3-9271-d8d42ed00d0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:10.631313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1260d81-7110-4ec3-9271-d8d42ed00d0d None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:10.653222 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 395/785] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 31 19:06:10 2022] GET /identity/v3/services => generated 772 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:10.660247 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0ed09-2f70-49b9-9b9d-1d08a7753290 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:10.692350 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1260d81-7110-4ec3-9271-d8d42ed00d0d None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:10.701366 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578bd9b-0b43-45c9-89df-e396b4833fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:10.702988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578bd9b-0b43-45c9-89df-e396b4833fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:10.703273 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578bd9b-0b43-45c9-89df-e396b4833fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:10.703550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578bd9b-0b43-45c9-89df-e396b4833fa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:10.704245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0578bd9b-0b43-45c9-89df-e396b4833fa4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:10.704550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0578bd9b-0b43-45c9-89df-e396b4833fa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:10.706531 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0578bd9b-0b43-45c9-89df-e396b4833fa4 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:10.716870 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 391/786] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:06:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:10.723877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578bd9b-0b43-45c9-89df-e396b4833fa4 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:10.799934 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:10.842931 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578bd9b-0b43-45c9-89df-e396b4833fa4 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:10.884196 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616512b1-887a-4ad8-9406-e2a823ac6a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:10.884853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616512b1-887a-4ad8-9406-e2a823ac6a79 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:10.885179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616512b1-887a-4ad8-9406-e2a823ac6a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:10.885456 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616512b1-887a-4ad8-9406-e2a823ac6a79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:10.899091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616512b1-887a-4ad8-9406-e2a823ac6a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:10.900546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616512b1-887a-4ad8-9406-e2a823ac6a79 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:10.916677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-616512b1-887a-4ad8-9406-e2a823ac6a79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:10.916677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:10.916677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:10.916677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:10.916677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:10.916677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:10.916677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:10.916677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:10.916677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:10.916677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:10.916677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:10.916677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:10.916677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:10.916677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:10.916677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:10.916677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:10.917708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:10.917708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:10.917708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:10.917708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:10.917708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:10.917708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:10.917708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:10.917708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:10.917708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:10.917708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:10.917708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:10.917708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:10.917708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:10.917708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:10.917708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:10.917708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:10.917708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:10.918651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:10.918651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:10.918651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:10.918651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 392/787] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:10.925309 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616512b1-887a-4ad8-9406-e2a823ac6a79 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:10.931391 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1260d81-7110-4ec3-9271-d8d42ed00d0d None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:10.979602 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70024f-eef8-4c5a-a8c2-ef98b4b2fe8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:10.980878 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70024f-eef8-4c5a-a8c2-ef98b4b2fe8b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:10.981883 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70024f-eef8-4c5a-a8c2-ef98b4b2fe8b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:10.982383 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70024f-eef8-4c5a-a8c2-ef98b4b2fe8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:10.998492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70024f-eef8-4c5a-a8c2-ef98b4b2fe8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:11.006285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-2d70024f-eef8-4c5a-a8c2-ef98b4b2fe8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:06:11.006285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:11.006285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:11.006285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:11.006285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:11.006285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:11.006285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:11.006285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:11.006285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:11.006285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:11.006285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:11.006285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:11.006285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:11.006285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:11.006285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:11.006285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:11.007524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:11.007524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:11.007524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:11.007524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:11.007524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:11.007524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:11.007524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:11.007524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:11.007524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:11.007524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:11.007524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:11.007524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:11.007524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:11.007524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:11.007524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:11.007524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:11.008623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:11.008623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:11.008623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:11.008623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:11.008623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:11.008623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:11.008623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:11.008623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:11.008623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:11.008623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:11.008623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:06:11.008623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:11.011274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 396/788] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:06:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:11.017215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70024f-eef8-4c5a-a8c2-ef98b4b2fe8b None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:11.045938 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:11.059912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616512b1-887a-4ad8-9406-e2a823ac6a79 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:11.065891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:11.118242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441677e-e59a-455d-99d1-4fd80f61dde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:11.118854 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441677e-e59a-455d-99d1-4fd80f61dde9 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:11.119157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441677e-e59a-455d-99d1-4fd80f61dde9 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:11.119426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441677e-e59a-455d-99d1-4fd80f61dde9 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:11.120072 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441677e-e59a-455d-99d1-4fd80f61dde9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:11.120372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441677e-e59a-455d-99d1-4fd80f61dde9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:11.121735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441677e-e59a-455d-99d1-4fd80f61dde9 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:11.147487 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 393/789] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:11.149763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70024f-eef8-4c5a-a8c2-ef98b4b2fe8b None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:11.153136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441677e-e59a-455d-99d1-4fd80f61dde9 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:11.215640 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04682ad6-3f6e-4f40-bc5c-186f043d4423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:11.216273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04682ad6-3f6e-4f40-bc5c-186f043d4423 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:11.216559 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04682ad6-3f6e-4f40-bc5c-186f043d4423 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:11.216840 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04682ad6-3f6e-4f40-bc5c-186f043d4423 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:11.217788 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04682ad6-3f6e-4f40-bc5c-186f043d4423 None admin] RBAC: Adding query filter params (name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:11.218100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04682ad6-3f6e-4f40-bc5c-186f043d4423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:11.219627 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04682ad6-3f6e-4f40-bc5c-186f043d4423 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:11.229429 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 397/790] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:06:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:11.237913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04682ad6-3f6e-4f40-bc5c-186f043d4423 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:11.337857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441677e-e59a-455d-99d1-4fd80f61dde9 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:11.395599 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcf10c-b29a-448f-913f-4457aac2d528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:11.396252 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcf10c-b29a-448f-913f-4457aac2d528 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:11.396532 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcf10c-b29a-448f-913f-4457aac2d528 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:11.396855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcf10c-b29a-448f-913f-4457aac2d528 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:11.404232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04682ad6-3f6e-4f40-bc5c-186f043d4423 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:11.404753 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcf10c-b29a-448f-913f-4457aac2d528 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:11.406193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcf10c-b29a-448f-913f-4457aac2d528 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:11.407033 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:11.419775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-59dcf10c-b29a-448f-913f-4457aac2d528 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:06:11.419775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:11.419775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:11.419775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:11.419775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:11.419775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:11.419775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:11.419775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:11.419775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:11.419775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:11.419775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:11.419775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:11.419775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:06:11.419775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:06:11.419775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:06:11.419775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:11.420802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:11.421793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:11.421793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:11.421793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:06:11.421793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:06:11.421793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:06:11.421793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:06:11.421793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:06:11.421793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:06:11.421793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:06:11.421793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:11.421793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 394/791] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:06:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:11.440015 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcf10c-b29a-448f-913f-4457aac2d528 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:11.447217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df54fc1-b0b2-47a3-9325-ef1470441395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:11.447863 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df54fc1-b0b2-47a3-9325-ef1470441395 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:11.448143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df54fc1-b0b2-47a3-9325-ef1470441395 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:11.448424 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df54fc1-b0b2-47a3-9325-ef1470441395 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:11.460696 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df54fc1-b0b2-47a3-9325-ef1470441395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:11.462167 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df54fc1-b0b2-47a3-9325-ef1470441395 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:11.469094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:11.478411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-4df54fc1-b0b2-47a3-9325-ef1470441395 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:11.478411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:11.478411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:11.478411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:11.478411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:11.478411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:11.478411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:11.478411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:11.478411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:11.478411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:11.478411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:11.478411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:11.478411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:11.478411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:11.478411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:11.478411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:11.483632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:11.483632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:11.483632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:11.483632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:11.483632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:11.483632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:11.483632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:11.483632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:11.483632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:11.483632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:11.483632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:11.483632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:11.483632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:11.483632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:11.483632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:11.483632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:11.483632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:11.484560 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:11.484560 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:11.484560 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:11.484560 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 398/792] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:11.485727 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8e427-8e5f-4335-b9cc-242e13d1c21c None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:11.486223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8e427-8e5f-4335-b9cc-242e13d1c21c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:11.487015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8e427-8e5f-4335-b9cc-242e13d1c21c None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:11.489514 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 399/793] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:11.490271 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8e427-8e5f-4335-b9cc-242e13d1c21c None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:11.584660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcf10c-b29a-448f-913f-4457aac2d528 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:11.624143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8e427-8e5f-4335-b9cc-242e13d1c21c None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:11.626132 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e1003-15f7-4d75-aa76-de00b2a1f1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:11.627036 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e1003-15f7-4d75-aa76-de00b2a1f1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:11.628306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e1003-15f7-4d75-aa76-de00b2a1f1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:11.628946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e1003-15f7-4d75-aa76-de00b2a1f1c7 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:11.629864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e1003-15f7-4d75-aa76-de00b2a1f1c7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:11.630391 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e1003-15f7-4d75-aa76-de00b2a1f1c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:11.632152 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e1003-15f7-4d75-aa76-de00b2a1f1c7 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:11.656835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 395/794] 10.222.0.59 () {56 vars in 1135 bytes} [Wed Aug 31 19:06:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:11.658745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9bfe6-45b3-4927-86b9-99d4cb0e2e69 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:11.659533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9bfe6-45b3-4927-86b9-99d4cb0e2e69 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:11.660109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9bfe6-45b3-4927-86b9-99d4cb0e2e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:11.663261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadcff5-515a-4196-bd86-066bbb4e1499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:11.663875 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadcff5-515a-4196-bd86-066bbb4e1499 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:11.664162 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadcff5-515a-4196-bd86-066bbb4e1499 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:11.664440 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadcff5-515a-4196-bd86-066bbb4e1499 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:11.665090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadcff5-515a-4196-bd86-066bbb4e1499 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:11.665396 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadcff5-515a-4196-bd86-066bbb4e1499 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:11.666770 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadcff5-515a-4196-bd86-066bbb4e1499 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:11.682276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 400/795] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:11.683201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadcff5-515a-4196-bd86-066bbb4e1499 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:11.684210 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-dda9bfe6-45b3-4927-86b9-99d4cb0e2e69 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:11.685718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-dda9bfe6-45b3-4927-86b9-99d4cb0e2e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:11.706558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:11.759500 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:11.766397 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 396/796] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:11.767562 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9bfe6-45b3-4927-86b9-99d4cb0e2e69 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:11.817452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadcff5-515a-4196-bd86-066bbb4e1499 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:11.883281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9bfe6-45b3-4927-86b9-99d4cb0e2e69 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:11.922403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a244e44-ea1d-4d65-a91b-d18c70840b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:11.923521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a244e44-ea1d-4d65-a91b-d18c70840b59 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:11.924062 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a244e44-ea1d-4d65-a91b-d18c70840b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:11.924918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2418-5337-46c9-9295-82ebf9877834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:11.925531 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a244e44-ea1d-4d65-a91b-d18c70840b59 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:11.925684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2418-5337-46c9-9295-82ebf9877834 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:11.925909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2418-5337-46c9-9295-82ebf9877834 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:11.926187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2418-5337-46c9-9295-82ebf9877834 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:11.934179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a244e44-ea1d-4d65-a91b-d18c70840b59 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:11.935962 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a244e44-ea1d-4d65-a91b-d18c70840b59 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:11.946106 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf2418-5337-46c9-9295-82ebf9877834 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:11.950521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-6a244e44-ea1d-4d65-a91b-d18c70840b59 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:06:11.950521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:11.950521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:11.950521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:11.950521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:11.950521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:11.950521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:11.950521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:11.950521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:11.950521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:11.950521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:11.950521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:11.950521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:06:11.950521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:06:11.950521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:06:11.950521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:11.951469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:11.952454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:11.952454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:11.952454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:06:11.952454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:06:11.952454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:06:11.952454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:06:11.952454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:06:11.952454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:06:11.952454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:06:11.952454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:11.954955 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 397/797] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 31 19:06:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:11.956373 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf2418-5337-46c9-9295-82ebf9877834 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:11.961264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a244e44-ea1d-4d65-a91b-d18c70840b59 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:11.994006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-33bf2418-5337-46c9-9295-82ebf9877834 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:11.994006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:11.994006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:11.994006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:11.994006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:11.994006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:11.994006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:11.994006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:11.994006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:11.994006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:11.994006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:11.994006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:11.994006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:11.994006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:11.994006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:11.994006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:11.995079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:11.995079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:11.995079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:11.995079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:11.995079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:11.995079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:11.995079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:11.995079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:11.995079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:11.995079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:11.995079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:11.995079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:11.995079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:11.995079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:11.995079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:11.995079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:11.995079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:11.996045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:11.996045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:11.996045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:11.997908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 401/798] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:12.004243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2418-5337-46c9-9295-82ebf9877834 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:12.062167 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:12.106808 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a244e44-ea1d-4d65-a91b-d18c70840b59 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:12.108882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:12.145556 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542282a-6792-4cc9-9409-181ff6e59e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:12.146096 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542282a-6792-4cc9-9409-181ff6e59e61 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:12.146378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542282a-6792-4cc9-9409-181ff6e59e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:12.146649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542282a-6792-4cc9-9409-181ff6e59e61 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:12.147854 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542282a-6792-4cc9-9409-181ff6e59e61 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:12.148177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542282a-6792-4cc9-9409-181ff6e59e61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:12.149607 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542282a-6792-4cc9-9409-181ff6e59e61 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:12.158300 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2418-5337-46c9-9295-82ebf9877834 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:12.168641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 398/799] 10.222.0.59 () {56 vars in 1149 bytes} [Wed Aug 31 19:06:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:12.170349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1975ca-110e-469d-ae9b-f57ed5e08ef7 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:12.170349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1975ca-110e-469d-ae9b-f57ed5e08ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:12.171044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1975ca-110e-469d-ae9b-f57ed5e08ef7 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:12.171989 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 399/800] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:12.177267 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1975ca-110e-469d-ae9b-f57ed5e08ef7 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:12.200197 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24379993-8b28-4883-be8b-fdc0718385de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:12.201072 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24379993-8b28-4883-be8b-fdc0718385de None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:12.201358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24379993-8b28-4883-be8b-fdc0718385de None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:12.201638 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24379993-8b28-4883-be8b-fdc0718385de None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:12.202286 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24379993-8b28-4883-be8b-fdc0718385de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:12.202591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24379993-8b28-4883-be8b-fdc0718385de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:12.204022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24379993-8b28-4883-be8b-fdc0718385de None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:12.216463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 402/801] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:12.217290 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24379993-8b28-4883-be8b-fdc0718385de None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:12.302543 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1975ca-110e-469d-ae9b-f57ed5e08ef7 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:12.322608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:12.325745 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24379993-8b28-4883-be8b-fdc0718385de None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:12.342762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422582bf-0978-47fc-8a82-26bbb6492e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:12.343689 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422582bf-0978-47fc-8a82-26bbb6492e40 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:12.344034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422582bf-0978-47fc-8a82-26bbb6492e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:12.344535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422582bf-0978-47fc-8a82-26bbb6492e40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:12.354704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:12.356402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422582bf-0978-47fc-8a82-26bbb6492e40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:12.362637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-422582bf-0978-47fc-8a82-26bbb6492e40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:12.362637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:12.362637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:12.362637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:12.362637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:12.362637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:12.362637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:12.362637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:12.362637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:12.362637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:12.362637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:12.362637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:12.362637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:12.362637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:12.362637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:12.362637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:12.368045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:12.368045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:12.368045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:12.368045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:12.368045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:12.368045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:12.368045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:12.368045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:12.368045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:12.368045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:12.368045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:12.368045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:12.368045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:12.368045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:12.368045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:12.368045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:12.369111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:12.369111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:12.369111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:12.369111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:12.369111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:12.369111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:12.369111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:12.369111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:12.369111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:12.369111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:12.369111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:12.369111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:12.369111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 400/802] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:06:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:12.369966 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07ee6a-2f07-4adb-b384-b049805cb342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:12.369966 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07ee6a-2f07-4adb-b384-b049805cb342 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:12.369966 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07ee6a-2f07-4adb-b384-b049805cb342 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:12.370223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec493b1b-3d88-4585-80d9-0a4a9e7d0184 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:12.370308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07ee6a-2f07-4adb-b384-b049805cb342 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:12.370390 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec493b1b-3d88-4585-80d9-0a4a9e7d0184 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:12.370390 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec493b1b-3d88-4585-80d9-0a4a9e7d0184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:12.382242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07ee6a-2f07-4adb-b384-b049805cb342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:12.383927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07ee6a-2f07-4adb-b384-b049805cb342 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:12.389627 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ec493b1b-3d88-4585-80d9-0a4a9e7d0184 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:12.390808 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ec493b1b-3d88-4585-80d9-0a4a9e7d0184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:12.401492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-0e07ee6a-2f07-4adb-b384-b049805cb342 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:12.401492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:12.401492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:12.401492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:12.401492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:12.401492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:12.401492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:12.401492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:12.401492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:12.401492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:12.401492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:12.401492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:12.401492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:12.401492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:12.401492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:12.401492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:12.402650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:12.402650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:12.402650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:12.402650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:12.402650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:12.402650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:12.402650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:12.402650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:12.402650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:12.402650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:12.402650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:12.402650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:12.402650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:12.402650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:12.402650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:12.402650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:12.402650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:12.403789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:12.403789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:12.403789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:12.409570 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 403/803] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:12.409570 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07ee6a-2f07-4adb-b384-b049805cb342 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:12.469251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 401/804] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:12.470231 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec493b1b-3d88-4585-80d9-0a4a9e7d0184 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:12.559427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07ee6a-2f07-4adb-b384-b049805cb342 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:12.601115 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57749b-52cf-4d1f-bef0-36f8b2988263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:12.601786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57749b-52cf-4d1f-bef0-36f8b2988263 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:12.602074 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57749b-52cf-4d1f-bef0-36f8b2988263 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:12.602354 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57749b-52cf-4d1f-bef0-36f8b2988263 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:12.610491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57749b-52cf-4d1f-bef0-36f8b2988263 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:12.612387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57749b-52cf-4d1f-bef0-36f8b2988263 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:12.625445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-4c57749b-52cf-4d1f-bef0-36f8b2988263 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:12.625445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:12.625445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:12.625445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:12.625445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:12.625445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:12.625445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:12.625445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:12.625445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:12.625445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:12.625445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:12.625445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:12.625445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:06:12.625445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:06:12.625445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:06:12.626578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:06:12.626578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:12.626578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:12.626578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:12.626578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:12.626578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:12.626578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:12.626578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:12.626578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:12.626578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:12.626578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:12.626578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:12.626578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:12.626578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:12.626578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:12.626578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:12.626578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:12.628070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:06:12.628070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:06:12.628070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:06:12.628070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:06:12.628070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:06:12.628070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:06:12.628070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:12.628070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:12.628070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 404/805] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:12.628568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57749b-52cf-4d1f-bef0-36f8b2988263 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:12.681274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:12.720005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec493b1b-3d88-4585-80d9-0a4a9e7d0184 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:12.729835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:12.767105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57749b-52cf-4d1f-bef0-36f8b2988263 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:12.769723 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f2853-d23d-4af6-89b8-abefe56745a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:12.770344 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f2853-d23d-4af6-89b8-abefe56745a2 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:12.770622 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f2853-d23d-4af6-89b8-abefe56745a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:12.770921 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f2853-d23d-4af6-89b8-abefe56745a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:12.771597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f2853-d23d-4af6-89b8-abefe56745a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:12.771905 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f2853-d23d-4af6-89b8-abefe56745a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:12.773268 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f2853-d23d-4af6-89b8-abefe56745a2 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:12.781684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 402/806] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:06:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:12.782483 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f2853-d23d-4af6-89b8-abefe56745a2 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:12.813659 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8791cd-bc8b-4795-a96c-1b8fd29ac0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:12.814276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8791cd-bc8b-4795-a96c-1b8fd29ac0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:12.814580 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8791cd-bc8b-4795-a96c-1b8fd29ac0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:12.815089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8791cd-bc8b-4795-a96c-1b8fd29ac0e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:12.815959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8791cd-bc8b-4795-a96c-1b8fd29ac0e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:12.816297 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8791cd-bc8b-4795-a96c-1b8fd29ac0e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:12.817703 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8791cd-bc8b-4795-a96c-1b8fd29ac0e8 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:12.831043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 405/807] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:12.832687 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8791cd-bc8b-4795-a96c-1b8fd29ac0e8 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:12.898813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f2853-d23d-4af6-89b8-abefe56745a2 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:12.934391 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d5761-cdfa-405e-8cc3-f590c05ddfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:12.934951 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d5761-cdfa-405e-8cc3-f590c05ddfdc None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:12.935236 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d5761-cdfa-405e-8cc3-f590c05ddfdc None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:12.935507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d5761-cdfa-405e-8cc3-f590c05ddfdc None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:12.936393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d5761-cdfa-405e-8cc3-f590c05ddfdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:12.936706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d5761-cdfa-405e-8cc3-f590c05ddfdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:12.938132 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d5761-cdfa-405e-8cc3-f590c05ddfdc None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:12.944436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:12.947348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8791cd-bc8b-4795-a96c-1b8fd29ac0e8 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:12.950206 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 403/808] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:06:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:12.951037 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d5761-cdfa-405e-8cc3-f590c05ddfdc None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:12.993938 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:13.079328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d5761-cdfa-405e-8cc3-f590c05ddfdc None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:13.106835 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da906334-fa50-47f9-8fc6-739c0c192b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:13.107524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da906334-fa50-47f9-8fc6-739c0c192b6f None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:13.107857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da906334-fa50-47f9-8fc6-739c0c192b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:13.108126 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da906334-fa50-47f9-8fc6-739c0c192b6f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:13.109043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da906334-fa50-47f9-8fc6-739c0c192b6f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:13.110100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da906334-fa50-47f9-8fc6-739c0c192b6f None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:13.117080 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00842d7d-a033-4afa-a53e-5c840aeea72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:13.117720 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00842d7d-a033-4afa-a53e-5c840aeea72b None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:13.118001 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00842d7d-a033-4afa-a53e-5c840aeea72b None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:13.118280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00842d7d-a033-4afa-a53e-5c840aeea72b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:13.120045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-da906334-fa50-47f9-8fc6-739c0c192b6f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:06:13.120045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:13.120045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:13.120045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:13.120045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:13.120045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:13.120045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:13.120045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:13.120045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:06:13.120045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:13.120045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:13.120045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:13.120045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:13.120045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:13.120045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:13.120045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:13.120045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:13.121410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:13.121410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:13.121410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:13.121410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:13.121410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:13.121410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:13.121410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:13.121410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:13.121410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:13.121410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:13.121410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:13.121410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:13.121410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:13.121410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:13.121410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:13.121410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:13.122371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:13.122371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:13.122371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:13.122371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:13.122371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:13.122371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:13.122371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:13.122371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:13.122371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:13.122371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:13.122371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:13.122371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:13.122371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:06:13.122371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:13.122371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 406/809] 10.222.0.59 () {56 vars in 1129 bytes} [Wed Aug 31 19:06:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:13.123239 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da906334-fa50-47f9-8fc6-739c0c192b6f None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:13.135682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00842d7d-a033-4afa-a53e-5c840aeea72b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:13.137134 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00842d7d-a033-4afa-a53e-5c840aeea72b None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:13.152819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-00842d7d-a033-4afa-a53e-5c840aeea72b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:13.152819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:13.152819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:13.152819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:13.152819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:13.152819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:13.152819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:13.152819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:13.152819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:13.152819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:13.152819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:13.152819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:13.152819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:13.152819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:13.152819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:13.152819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:13.154612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:13.154612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:13.154612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:13.154612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:13.154612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:13.154612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:13.154612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:13.154612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:13.154612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:13.154612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:13.154612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:13.154612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:13.154612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:13.154612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:13.154612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:13.154612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:13.154612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:13.155594 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:13.155594 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:13.155594 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:13.155594 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 404/810] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:13.155594 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366e94d-93a4-4ffc-9e35-69af4f51bd8d None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:13.155892 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366e94d-93a4-4ffc-9e35-69af4f51bd8d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:13.156039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366e94d-93a4-4ffc-9e35-69af4f51bd8d None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:13.157271 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 405/811] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:13.157947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366e94d-93a4-4ffc-9e35-69af4f51bd8d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:13.240710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da906334-fa50-47f9-8fc6-739c0c192b6f None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:13.252211 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:13.281295 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366e94d-93a4-4ffc-9e35-69af4f51bd8d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:13.282264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddf12a-93df-4f64-9804-80b0b5a45932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:13.282925 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddf12a-93df-4f64-9804-80b0b5a45932 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:13.283263 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddf12a-93df-4f64-9804-80b0b5a45932 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:13.283556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddf12a-93df-4f64-9804-80b0b5a45932 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:13.290829 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddf12a-93df-4f64-9804-80b0b5a45932 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:13.292296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddf12a-93df-4f64-9804-80b0b5a45932 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:13.304411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-c8ddf12a-93df-4f64-9804-80b0b5a45932 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:13.304411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:13.304411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:13.304411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:13.304411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:13.304411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:13.304411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:13.304411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:13.304411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:13.304411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:13.304411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:13.304411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:13.304411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:06:13.304411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:06:13.304411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:06:13.306595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:06:13.306595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:13.306595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:13.306595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:13.306595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:13.306595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:13.306595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:13.306595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:13.306595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:13.306595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:13.306595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:13.306595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:13.306595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:13.306595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:13.306595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:13.306595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:13.306595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:13.308620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:06:13.308620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:06:13.308620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:06:13.308620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:06:13.308620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:06:13.308620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:06:13.308620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:13.308620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:13.308620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 407/812] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:13.308620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddf12a-93df-4f64-9804-80b0b5a45932 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:13.330792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff586f-6a69-4fe4-9f1a-43e4029d3e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:13.331482 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff586f-6a69-4fe4-9f1a-43e4029d3e72 None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:13.331767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff586f-6a69-4fe4-9f1a-43e4029d3e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:13.332037 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff586f-6a69-4fe4-9f1a-43e4029d3e72 None admin] PATH_INFO: `/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/25ed58369ffd46e8b7fa1880662926e8/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:13.333918 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff586f-6a69-4fe4-9f1a-43e4029d3e72 None admin] RBAC: Authorizing `identity:create_grant(project_id=161c90d34bb34925a1aaeb70c7de7eae, user_id=25ed58369ffd46e8b7fa1880662926e8, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:13.335260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:13.335376 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff586f-6a69-4fe4-9f1a-43e4029d3e72 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:13.402472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 406/813] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:06:13 2022] PUT /identity/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/25ed58369ffd46e8b7fa1880662926e8/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:06:13.404120 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883be0e-2b54-4559-861b-a8fb2ffe10c6 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:13.404462 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883be0e-2b54-4559-861b-a8fb2ffe10c6 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:13.404816 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883be0e-2b54-4559-861b-a8fb2ffe10c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:13.426257 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddf12a-93df-4f64-9804-80b0b5a45932 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:13.429886 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-d883be0e-2b54-4559-861b-a8fb2ffe10c6 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:13.431537 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d883be0e-2b54-4559-861b-a8fb2ffe10c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:13.474977 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9050d-970b-4d91-ace9-2974a8f1caff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:13.476691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9050d-970b-4d91-ace9-2974a8f1caff None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:13.480274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9050d-970b-4d91-ace9-2974a8f1caff None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:13.480274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9050d-970b-4d91-ace9-2974a8f1caff None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:13.480274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9050d-970b-4d91-ace9-2974a8f1caff None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:13.480274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9050d-970b-4d91-ace9-2974a8f1caff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:13.480274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9050d-970b-4d91-ace9-2974a8f1caff None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:13.494548 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 408/814] 10.222.0.59 () {56 vars in 1124 bytes} [Wed Aug 31 19:06:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:13.494548 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9050d-970b-4d91-ace9-2974a8f1caff None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:13.526642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 407/815] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:13.531893 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883be0e-2b54-4559-861b-a8fb2ffe10c6 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:13.612917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:13.632249 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9050d-970b-4d91-ace9-2974a8f1caff None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:13.638322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:13.674712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c8001-df39-44fb-a8e3-9047da8f5036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:13.675815 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c8001-df39-44fb-a8e3-9047da8f5036 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:13.676342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c8001-df39-44fb-a8e3-9047da8f5036 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:13.676720 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c8001-df39-44fb-a8e3-9047da8f5036 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:13.677503 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c8001-df39-44fb-a8e3-9047da8f5036 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:13.677907 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c8001-df39-44fb-a8e3-9047da8f5036 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:13.679510 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c8001-df39-44fb-a8e3-9047da8f5036 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:13.683504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883be0e-2b54-4559-861b-a8fb2ffe10c6 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:13.698283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 409/816] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:13.699234 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c8001-df39-44fb-a8e3-9047da8f5036 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:13.730260 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb15b7f-fc7c-431c-8540-dd2952e1a4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:13.730948 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb15b7f-fc7c-431c-8540-dd2952e1a4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:13.731227 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb15b7f-fc7c-431c-8540-dd2952e1a4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:13.731548 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb15b7f-fc7c-431c-8540-dd2952e1a4e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:13.733323 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb15b7f-fc7c-431c-8540-dd2952e1a4e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:13.733652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb15b7f-fc7c-431c-8540-dd2952e1a4e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:13.735258 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb15b7f-fc7c-431c-8540-dd2952e1a4e9 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:13.753680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 408/817] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:06:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:13.754544 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb15b7f-fc7c-431c-8540-dd2952e1a4e9 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:13.810549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c8001-df39-44fb-a8e3-9047da8f5036 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:13.850615 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06369dc0-cb7d-4a91-98aa-0f544508beea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:13.851275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06369dc0-cb7d-4a91-98aa-0f544508beea None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:13.851546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06369dc0-cb7d-4a91-98aa-0f544508beea None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:13.851828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06369dc0-cb7d-4a91-98aa-0f544508beea None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:13.852461 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06369dc0-cb7d-4a91-98aa-0f544508beea None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:13.852764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06369dc0-cb7d-4a91-98aa-0f544508beea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:13.854133 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06369dc0-cb7d-4a91-98aa-0f544508beea None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:13.859689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 410/818] 10.222.0.59 () {56 vars in 1124 bytes} [Wed Aug 31 19:06:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:13.861009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06369dc0-cb7d-4a91-98aa-0f544508beea None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:13.882400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:13.889593 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb15b7f-fc7c-431c-8540-dd2952e1a4e9 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:13.920840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:13.928312 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8e39c-0998-4531-be44-16da07989e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:13.928312 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8e39c-0998-4531-be44-16da07989e28 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:13.928312 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8e39c-0998-4531-be44-16da07989e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:13.928312 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8e39c-0998-4531-be44-16da07989e28 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:13.937817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8e39c-0998-4531-be44-16da07989e28 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:13.939307 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8e39c-0998-4531-be44-16da07989e28 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:13.951434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-08e8e39c-0998-4531-be44-16da07989e28 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:06:13.951434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:13.951434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:13.951434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:13.951434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:13.951434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:13.951434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:13.951434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:13.951434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:13.951434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:13.951434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:13.951434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:13.951434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:06:13.951434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:06:13.951434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:06:13.951434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:13.954707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:13.956737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:13.956737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:13.956737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:06:13.956737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:06:13.956737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:06:13.956737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:06:13.956737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:06:13.956737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:06:13.956737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:06:13.956737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:13.970804 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 409/819] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 31 19:06:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:13.973059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8e39c-0998-4531-be44-16da07989e28 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:13.997304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06369dc0-cb7d-4a91-98aa-0f544508beea None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:14.034820 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd01c11-2516-4cca-b8a6-8b2707958686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:14.035486 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd01c11-2516-4cca-b8a6-8b2707958686 None admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:14.035769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd01c11-2516-4cca-b8a6-8b2707958686 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:14.036042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd01c11-2516-4cca-b8a6-8b2707958686 None admin] PATH_INFO: `/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/1ee990b1658e49cc8bc2fe15e7dc36d9/roles/a9ef0098e982437fb0127827fa2cad38` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:14.051579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd01c11-2516-4cca-b8a6-8b2707958686 None admin] RBAC: Authorizing `identity:create_grant(project_id=161c90d34bb34925a1aaeb70c7de7eae, user_id=1ee990b1658e49cc8bc2fe15e7dc36d9, role_id=a9ef0098e982437fb0127827fa2cad38)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:14.053041 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd01c11-2516-4cca-b8a6-8b2707958686 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:14.098237 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8e39c-0998-4531-be44-16da07989e28 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:14.139400 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaccda-fc27-4d08-853e-e2f2855b181e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:14.140057 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaccda-fc27-4d08-853e-e2f2855b181e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:14.140337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaccda-fc27-4d08-853e-e2f2855b181e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:14.140610 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaccda-fc27-4d08-853e-e2f2855b181e None admin] PATH_INFO: `/v3/services/network` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:14.141214 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeaccda-fc27-4d08-853e-e2f2855b181e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:14.143606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 411/820] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:06:13 2022] PUT /identity/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/1ee990b1658e49cc8bc2fe15e7dc36d9/roles/a9ef0098e982437fb0127827fa2cad38 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:06:14.143808 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeaccda-fc27-4d08-853e-e2f2855b181e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:14.144607 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd01c11-2516-4cca-b8a6-8b2707958686 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:14.156735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-4aeaccda-fc27-4d08-853e-e2f2855b181e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:06:14.156735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:14.156735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:14.156735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:14.156735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:14.156735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:14.156735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:14.156735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:14.156735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:06:14.156735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:14.156735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:14.156735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:14.156735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:14.156735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:14.156735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:14.156735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:14.156735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:14.157839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:14.157839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:14.157839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:14.157839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:14.157839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:14.157839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:14.157839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:14.157839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:14.157839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:14.157839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:14.157839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:14.157839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:14.157839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:14.157839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:14.157839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:14.157839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:14.159156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:14.159156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:14.159156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:14.159156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:14.159156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:14.159156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:14.159156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:14.159156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:14.159156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:14.159156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:14.159156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:14.159156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:14.159156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:06:14.159156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:14.165178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 410/821] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:13 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:14.166286 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaccda-fc27-4d08-853e-e2f2855b181e None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:14.252522 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:14.261622 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:14.310208 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd01c11-2516-4cca-b8a6-8b2707958686 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:14.327869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaccda-fc27-4d08-853e-e2f2855b181e None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:14.355611 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605528a-f45a-496b-8e56-cdc2a48d65ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:14.356231 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605528a-f45a-496b-8e56-cdc2a48d65ed None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:14.356513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605528a-f45a-496b-8e56-cdc2a48d65ed None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:14.356786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605528a-f45a-496b-8e56-cdc2a48d65ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:14.357464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d605528a-f45a-496b-8e56-cdc2a48d65ed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:14.358973 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d605528a-f45a-496b-8e56-cdc2a48d65ed None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:14.376732 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65fb80-3cba-475e-8ec9-eca82b18fa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:14.378028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65fb80-3cba-475e-8ec9-eca82b18fa1a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:14.381069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65fb80-3cba-475e-8ec9-eca82b18fa1a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:14.381069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65fb80-3cba-475e-8ec9-eca82b18fa1a None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:14.381069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65fb80-3cba-475e-8ec9-eca82b18fa1a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:14.381069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65fb80-3cba-475e-8ec9-eca82b18fa1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:14.381397 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65fb80-3cba-475e-8ec9-eca82b18fa1a None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:14.401514 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 411/822] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 31 19:06:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:14.405069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65fb80-3cba-475e-8ec9-eca82b18fa1a None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:14.453836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 412/823] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:06:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:14.455618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605528a-f45a-496b-8e56-cdc2a48d65ed None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:14.592918 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65fb80-3cba-475e-8ec9-eca82b18fa1a None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:14.624798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:14.673050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605528a-f45a-496b-8e56-cdc2a48d65ed None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:14.676530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c9523-2cb1-4f9e-9bc0-e57b1d993c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:14.677173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c9523-2cb1-4f9e-9bc0-e57b1d993c6e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:14.677424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c9523-2cb1-4f9e-9bc0-e57b1d993c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:14.677695 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c9523-2cb1-4f9e-9bc0-e57b1d993c6e None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:14.678324 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c9523-2cb1-4f9e-9bc0-e57b1d993c6e None admin] RBAC: Adding query filter params (name=network) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:14.680190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c9523-2cb1-4f9e-9bc0-e57b1d993c6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:14.680190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c9523-2cb1-4f9e-9bc0-e57b1d993c6e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:14.692304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 412/824] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 31 19:06:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:14.700648 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:14.701102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c9523-2cb1-4f9e-9bc0-e57b1d993c6e None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:14.723461 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520fed7-5a65-412e-9501-5c74b85708e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:14.724134 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520fed7-5a65-412e-9501-5c74b85708e5 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:14.724416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520fed7-5a65-412e-9501-5c74b85708e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:14.724688 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520fed7-5a65-412e-9501-5c74b85708e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:14.815431 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520fed7-5a65-412e-9501-5c74b85708e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:14.816906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520fed7-5a65-412e-9501-5c74b85708e5 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:14.842393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c9523-2cb1-4f9e-9bc0-e57b1d993c6e None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:14.887641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca2b65-ada8-48b6-ab0e-1b0669929af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:14.888582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca2b65-ada8-48b6-ab0e-1b0669929af0 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:14.888582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca2b65-ada8-48b6-ab0e-1b0669929af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:14.888868 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca2b65-ada8-48b6-ab0e-1b0669929af0 None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:14.889826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca2b65-ada8-48b6-ab0e-1b0669929af0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:14.890332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca2b65-ada8-48b6-ab0e-1b0669929af0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:14.891737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca2b65-ada8-48b6-ab0e-1b0669929af0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:14.899737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 413/825] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 31 19:06:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:14.908919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca2b65-ada8-48b6-ab0e-1b0669929af0 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:14.952869 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 413/826] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:06:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:06:14.956202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:14.958390 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520fed7-5a65-412e-9501-5c74b85708e5 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:14.974927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:15.071475 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520fed7-5a65-412e-9501-5c74b85708e5 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:15.086842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca2b65-ada8-48b6-ab0e-1b0669929af0 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:15.113098 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea56b9c-2df2-4ae2-b891-f62aced7f714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:15.113748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea56b9c-2df2-4ae2-b891-f62aced7f714 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:15.114031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea56b9c-2df2-4ae2-b891-f62aced7f714 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:15.114305 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea56b9c-2df2-4ae2-b891-f62aced7f714 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:15.139126 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea56b9c-2df2-4ae2-b891-f62aced7f714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:15.140511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea56b9c-2df2-4ae2-b891-f62aced7f714 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:15.157769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77378700-6444-4e99-a558-31795e602803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:15.158458 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77378700-6444-4e99-a558-31795e602803 None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:15.158742 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77378700-6444-4e99-a558-31795e602803 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:15.159085 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77378700-6444-4e99-a558-31795e602803 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:15.159767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77378700-6444-4e99-a558-31795e602803 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:15.161389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77378700-6444-4e99-a558-31795e602803 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:15.175723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-fea56b9c-2df2-4ae2-b891-f62aced7f714 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:15.175723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:15.175723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:15.175723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:15.175723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:15.175723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:15.175723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:15.175723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:15.175723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:15.175723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:15.175723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:15.175723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:15.175723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:15.175723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:15.175723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:15.175723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:15.177676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:15.177676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:15.177676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:15.178854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 414/827] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:15.184892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea56b9c-2df2-4ae2-b891-f62aced7f714 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:15.277348 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 414/828] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:06:14 2022] POST /identity/v3/endpoints => generated 337 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:15.279211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:15.345318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea56b9c-2df2-4ae2-b891-f62aced7f714 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:15.410484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381df0d5-0b01-4a09-8b38-b0a0d526adce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:15.411119 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381df0d5-0b01-4a09-8b38-b0a0d526adce None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:15.411411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381df0d5-0b01-4a09-8b38-b0a0d526adce None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:15.411690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381df0d5-0b01-4a09-8b38-b0a0d526adce None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:15.412361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381df0d5-0b01-4a09-8b38-b0a0d526adce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:15.412665 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381df0d5-0b01-4a09-8b38-b0a0d526adce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:15.414073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381df0d5-0b01-4a09-8b38-b0a0d526adce None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:15.433637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 415/829] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:15.444396 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77378700-6444-4e99-a558-31795e602803 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:15.587245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:15.589026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77378700-6444-4e99-a558-31795e602803 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:15.619342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6750cb-fef3-47ef-80b4-46a9e26e364c None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:15.619817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6750cb-fef3-47ef-80b4-46a9e26e364c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:15.620218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6750cb-fef3-47ef-80b4-46a9e26e364c None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:15.621793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 416/830] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:15.628683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e884cd3-f065-411a-a577-5d3fbed66c60 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:15.629242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e884cd3-f065-411a-a577-5d3fbed66c60 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:15.629646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e884cd3-f065-411a-a577-5d3fbed66c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:15.633049 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba1c0e-5cc1-46d2-b7a8-d99e6246d57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:15.635565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba1c0e-5cc1-46d2-b7a8-d99e6246d57c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:15.635565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba1c0e-5cc1-46d2-b7a8-d99e6246d57c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:15.635565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba1c0e-5cc1-46d2-b7a8-d99e6246d57c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:15.642903 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba1c0e-5cc1-46d2-b7a8-d99e6246d57c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:15.644940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba1c0e-5cc1-46d2-b7a8-d99e6246d57c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:15.656029 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-7e884cd3-f065-411a-a577-5d3fbed66c60 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:15.657339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-0cba1c0e-5cc1-46d2-b7a8-d99e6246d57c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:15.657339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:15.657339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:15.657339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:15.657339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:15.657339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:15.657339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:15.657339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:15.657339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:15.657339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:15.657339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:15.657339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:15.657339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:06:15.657339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:06:15.657339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:06:15.658428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:06:15.658428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:15.658428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:15.658428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:15.658428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:15.658428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:15.658428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:15.658428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:15.658428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:15.658428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:15.658428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:15.658428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:15.658428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:15.658428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:15.658428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:15.658428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:15.658428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:15.659585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:06:15.659585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:06:15.659585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:06:15.659585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:06:15.659585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:06:15.659585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:06:15.659585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:15.659585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:15.660359 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 415/831] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:15.660693 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7e884cd3-f065-411a-a577-5d3fbed66c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:15.666506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba1c0e-5cc1-46d2-b7a8-d99e6246d57c None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:15.762771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 417/832] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:15.796954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba1c0e-5cc1-46d2-b7a8-d99e6246d57c None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:15.838089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06973bb5-de11-44fc-a7c1-44934496945c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:15.838705 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06973bb5-de11-44fc-a7c1-44934496945c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:15.839016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06973bb5-de11-44fc-a7c1-44934496945c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:15.839298 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06973bb5-de11-44fc-a7c1-44934496945c None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:15.840047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06973bb5-de11-44fc-a7c1-44934496945c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:15.840358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06973bb5-de11-44fc-a7c1-44934496945c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:15.841828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06973bb5-de11-44fc-a7c1-44934496945c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:15.858817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 416/833] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:06:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:15.865958 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e884cd3-f065-411a-a577-5d3fbed66c60 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:15.921003 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:15.971701 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e884cd3-f065-411a-a577-5d3fbed66c60 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:16.007383 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1081e-0620-4e9e-a73e-63ae225bf5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:16.007996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1081e-0620-4e9e-a73e-63ae225bf5b4 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:16.008274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1081e-0620-4e9e-a73e-63ae225bf5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:16.008548 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1081e-0620-4e9e-a73e-63ae225bf5b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:16.009276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1081e-0620-4e9e-a73e-63ae225bf5b4 None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=d461422ce98442bcbe6b254a8dad5e8a) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:16.009591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1081e-0620-4e9e-a73e-63ae225bf5b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:16.011043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1081e-0620-4e9e-a73e-63ae225bf5b4 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:16.017997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 418/834] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:06:15 2022] GET /identity/v3/role_assignments?user.id=d461422ce98442bcbe6b254a8dad5e8a&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:16.073103 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06973bb5-de11-44fc-a7c1-44934496945c None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:16.101930 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:16.203622 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06973bb5-de11-44fc-a7c1-44934496945c None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:16.206040 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2e51a-4487-43c7-8a6a-afcfe6448824 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:16.206614 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2e51a-4487-43c7-8a6a-afcfe6448824 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:16.207296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2e51a-4487-43c7-8a6a-afcfe6448824 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:16.208924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 419/835] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:16.214368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cd684-a031-401e-a510-e6a269e80f04 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:16.214625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cd684-a031-401e-a510-e6a269e80f04 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:16.214924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cd684-a031-401e-a510-e6a269e80f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:16.238409 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-e18cd684-a031-401e-a510-e6a269e80f04 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:16.239622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e18cd684-a031-401e-a510-e6a269e80f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:16.251028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464016b-cdc3-476c-8405-9696a10a550c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:16.252606 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464016b-cdc3-476c-8405-9696a10a550c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:16.253314 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464016b-cdc3-476c-8405-9696a10a550c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:16.254255 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464016b-cdc3-476c-8405-9696a10a550c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:16.269553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464016b-cdc3-476c-8405-9696a10a550c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:16.275866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-6464016b-cdc3-476c-8405-9696a10a550c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:16.275866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:16.275866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:16.275866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:16.275866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:16.275866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:16.275866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:16.275866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:16.275866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:16.275866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:16.275866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:16.275866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:16.275866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:16.275866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:16.275866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:16.275866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:16.277017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:16.277017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:16.277017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:16.277017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:16.277017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:16.277017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:16.277017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:16.277017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:16.277017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:16.277017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:16.277017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:16.277017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:16.277017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:16.277017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:16.277017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:16.277017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:16.278059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:16.278059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:16.278059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:16.278059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:16.278059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:16.278059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:16.278059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:16.278059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:16.278059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:16.278059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:16.278059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:16.278059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:16.278059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 417/836] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:06:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:16.288428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464016b-cdc3-476c-8405-9696a10a550c None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:16.339333 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 420/837] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:16.388665 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:16.415294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464016b-cdc3-476c-8405-9696a10a550c None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:16.452513 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9269b0-874b-460c-b3ae-12cc38691f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:16.453952 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9269b0-874b-460c-b3ae-12cc38691f40 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:16.454414 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9269b0-874b-460c-b3ae-12cc38691f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:16.454865 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9269b0-874b-460c-b3ae-12cc38691f40 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:16.456419 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9269b0-874b-460c-b3ae-12cc38691f40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:16.456913 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9269b0-874b-460c-b3ae-12cc38691f40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:16.458475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9269b0-874b-460c-b3ae-12cc38691f40 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:16.468810 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 418/838] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:06:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:16.475316 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cd684-a031-401e-a510-e6a269e80f04 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:16.561024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:16.615622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cd684-a031-401e-a510-e6a269e80f04 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:16.654427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321a6ac-02ab-497e-aea3-c080fab77818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:16.655448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321a6ac-02ab-497e-aea3-c080fab77818 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:16.655914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321a6ac-02ab-497e-aea3-c080fab77818 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:16.656389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321a6ac-02ab-497e-aea3-c080fab77818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:16.667563 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321a6ac-02ab-497e-aea3-c080fab77818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:16.669327 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321a6ac-02ab-497e-aea3-c080fab77818 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:16.686761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-6321a6ac-02ab-497e-aea3-c080fab77818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:16.686761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:16.686761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:16.686761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:16.686761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:16.686761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:16.686761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:16.686761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:16.686761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:16.686761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:16.686761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:16.686761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:16.686761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:16.686761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:16.686761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:16.686761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:16.687809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:16.687809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:16.687809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:16.687809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:16.687809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:16.687809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:16.687809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:16.687809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:16.687809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:16.687809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:16.687809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:16.687809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:16.687809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:16.687809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:16.687809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:16.687809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:16.687809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:16.688754 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:16.688754 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:16.688754 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:16.691504 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 421/839] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:16.700087 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9269b0-874b-460c-b3ae-12cc38691f40 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:16.742488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321a6ac-02ab-497e-aea3-c080fab77818 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:16.862834 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321a6ac-02ab-497e-aea3-c080fab77818 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:16.887255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:16.897241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9269b0-874b-460c-b3ae-12cc38691f40 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:16.906084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705d47a-aa4c-40c3-b971-c7cb7ac6e12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:16.907027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705d47a-aa4c-40c3-b971-c7cb7ac6e12c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:16.907323 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705d47a-aa4c-40c3-b971-c7cb7ac6e12c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:16.907598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705d47a-aa4c-40c3-b971-c7cb7ac6e12c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:16.920966 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705d47a-aa4c-40c3-b971-c7cb7ac6e12c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:16.928273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-3705d47a-aa4c-40c3-b971-c7cb7ac6e12c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:06:16.928273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:16.928273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:16.928273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:16.928273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:16.928273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:16.928273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:16.928273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:16.928273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:16.928273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:16.928273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:16.928273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:16.928273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:16.928273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:16.928273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:16.928273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:16.929301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:16.929301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:16.929301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:16.929301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:16.929301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:16.929301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:16.929301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:16.929301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:16.929301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:16.929301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:16.929301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:16.929301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:16.929301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:16.929301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:16.929301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:16.929301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:16.930359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:16.930359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:16.930359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:16.930359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:16.930359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:16.930359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:16.930359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:16.930359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:16.930359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:16.930359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:16.930359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:06:16.930359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:16.930359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 422/840] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:06:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:16.937246 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705d47a-aa4c-40c3-b971-c7cb7ac6e12c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:16.967026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd9bc0f-5db7-4be7-aa14-d344f1bb8dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:16.967564 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9bc0f-5db7-4be7-aa14-d344f1bb8dbe None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:16.967842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9bc0f-5db7-4be7-aa14-d344f1bb8dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:16.968106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9bc0f-5db7-4be7-aa14-d344f1bb8dbe None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:16.968762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd9bc0f-5db7-4be7-aa14-d344f1bb8dbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:16.969066 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd9bc0f-5db7-4be7-aa14-d344f1bb8dbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:16.971475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd9bc0f-5db7-4be7-aa14-d344f1bb8dbe None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:16.980201 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:16.996282 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 419/841] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:16.996282 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd9bc0f-5db7-4be7-aa14-d344f1bb8dbe None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:17.135888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd9bc0f-5db7-4be7-aa14-d344f1bb8dbe None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:17.148694 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705d47a-aa4c-40c3-b971-c7cb7ac6e12c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:17.181294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f3aa4-308d-4434-b567-d2336b3fdc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:17.182274 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f3aa4-308d-4434-b567-d2336b3fdc59 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:17.182727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f3aa4-308d-4434-b567-d2336b3fdc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:17.183207 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f3aa4-308d-4434-b567-d2336b3fdc59 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:17.191788 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f3aa4-308d-4434-b567-d2336b3fdc59 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:17.193368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f3aa4-308d-4434-b567-d2336b3fdc59 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:17.203030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b96889-9d57-4fba-bf6e-d9957e71c6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:17.203655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b96889-9d57-4fba-bf6e-d9957e71c6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:17.203938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b96889-9d57-4fba-bf6e-d9957e71c6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:17.204209 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b96889-9d57-4fba-bf6e-d9957e71c6a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:17.204895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b96889-9d57-4fba-bf6e-d9957e71c6a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:17.205242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b96889-9d57-4fba-bf6e-d9957e71c6a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:17.205992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-305f3aa4-308d-4434-b567-d2336b3fdc59 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:06:17.205992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:17.205992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:17.205992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:17.205992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:17.205992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:17.205992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:17.205992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:17.205992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:17.205992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:17.205992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:17.205992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:17.205992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:06:17.205992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:06:17.205992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:06:17.205992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:17.207490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:17.208864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:17.208864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:17.208864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:06:17.208864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:06:17.208864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:06:17.208864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:06:17.208864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:06:17.208864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:06:17.208864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:06:17.208864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:17.209414 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b96889-9d57-4fba-bf6e-d9957e71c6a2 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:17.209497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 420/842] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:06:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:17.209497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32ac36-1590-4ff6-a4d5-c4554881d505 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:17.209497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32ac36-1590-4ff6-a4d5-c4554881d505 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:17.209497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32ac36-1590-4ff6-a4d5-c4554881d505 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:17.210568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 421/843] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:17.215179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e32ac36-1590-4ff6-a4d5-c4554881d505 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:17.222041 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 423/844] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:06:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:17.227946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b3446-207a-48bb-9e9e-c6bf96167d34 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:17.228244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b3446-207a-48bb-9e9e-c6bf96167d34 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:17.228522 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b3446-207a-48bb-9e9e-c6bf96167d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:17.248873 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-c65b3446-207a-48bb-9e9e-c6bf96167d34 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:17.250058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c65b3446-207a-48bb-9e9e-c6bf96167d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:17.266954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:17.290354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:17.342639 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e32ac36-1590-4ff6-a4d5-c4554881d505 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:17.345122 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 424/845] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:17.352427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b3446-207a-48bb-9e9e-c6bf96167d34 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:17.383014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27995ca-4333-4882-904e-b1b0e37aaad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:17.383812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27995ca-4333-4882-904e-b1b0e37aaad1 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:17.384194 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27995ca-4333-4882-904e-b1b0e37aaad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:17.384568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27995ca-4333-4882-904e-b1b0e37aaad1 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:17.385647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27995ca-4333-4882-904e-b1b0e37aaad1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:17.385765 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27995ca-4333-4882-904e-b1b0e37aaad1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:17.387426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27995ca-4333-4882-904e-b1b0e37aaad1 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:17.411981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 422/846] 10.222.0.59 () {56 vars in 1135 bytes} [Wed Aug 31 19:06:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:17.413320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27995ca-4333-4882-904e-b1b0e37aaad1 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:17.490069 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b3446-207a-48bb-9e9e-c6bf96167d34 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:17.529093 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a11448-9880-4ccc-a6e0-6ca6d517900f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:17.529745 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a11448-9880-4ccc-a6e0-6ca6d517900f None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:17.530029 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a11448-9880-4ccc-a6e0-6ca6d517900f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:17.530303 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a11448-9880-4ccc-a6e0-6ca6d517900f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:17.542775 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a11448-9880-4ccc-a6e0-6ca6d517900f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:17.544379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a11448-9880-4ccc-a6e0-6ca6d517900f None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:17.556265 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27995ca-4333-4882-904e-b1b0e37aaad1 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:17.559897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-87a11448-9880-4ccc-a6e0-6ca6d517900f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:17.559897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:17.559897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:17.559897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:17.559897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:17.559897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:17.559897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:17.559897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:17.559897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:17.559897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:17.559897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:17.559897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:17.559897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:17.559897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:17.559897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:17.559897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:17.560856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:17.560856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:17.560856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:17.560856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:17.560856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:17.560856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:17.560856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:17.560856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:17.560856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:17.560856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:17.560856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:17.560856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:17.560856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:17.560856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:17.560856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:17.560856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:17.560856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:17.561773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:17.561773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:17.561773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:17.566064 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 425/847] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:17.567001 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a11448-9880-4ccc-a6e0-6ca6d517900f None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:17.570081 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:17.581673 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:17.609652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05703bc5-dcad-41f9-babc-7314466526a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:17.610225 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05703bc5-dcad-41f9-babc-7314466526a5 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:17.610509 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05703bc5-dcad-41f9-babc-7314466526a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:17.610777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05703bc5-dcad-41f9-babc-7314466526a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:17.687057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a11448-9880-4ccc-a6e0-6ca6d517900f None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:17.693867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05703bc5-dcad-41f9-babc-7314466526a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:17.695357 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05703bc5-dcad-41f9-babc-7314466526a5 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:17.725499 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056df615-7351-4839-b4fe-317aafd84936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:17.726120 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056df615-7351-4839-b4fe-317aafd84936 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:17.726422 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056df615-7351-4839-b4fe-317aafd84936 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:17.726697 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056df615-7351-4839-b4fe-317aafd84936 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:17.727388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056df615-7351-4839-b4fe-317aafd84936 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:17.727696 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056df615-7351-4839-b4fe-317aafd84936 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:17.729073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056df615-7351-4839-b4fe-317aafd84936 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:17.742795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 426/848] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:17.743672 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056df615-7351-4839-b4fe-317aafd84936 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:17.809896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 423/849] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:06:17 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:06:17.811034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05703bc5-dcad-41f9-babc-7314466526a5 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:17.816592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:17.831257 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:17.860587 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056df615-7351-4839-b4fe-317aafd84936 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:17.897936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8e952-944b-4477-ba95-1011a16872d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:17.898579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8e952-944b-4477-ba95-1011a16872d2 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:17.898856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8e952-944b-4477-ba95-1011a16872d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:17.899161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8e952-944b-4477-ba95-1011a16872d2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:17.899754 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8e952-944b-4477-ba95-1011a16872d2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:17.901124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8e952-944b-4477-ba95-1011a16872d2 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:17.911636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-d8e8e952-944b-4477-ba95-1011a16872d2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:06:17.911636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:17.911636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:17.911636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:17.911636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:17.911636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:17.911636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:17.911636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:17.911636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:06:17.911636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:17.911636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:17.911636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:17.911636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:17.911636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:17.911636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:17.911636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:17.911636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:17.913372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:17.913372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:17.913372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:17.913372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:17.913372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:17.913372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:17.913372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:17.913372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:17.913372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:17.913372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:17.913372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:17.913372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:17.913372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:17.913372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:17.913372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:17.913372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:17.914729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:17.914729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:17.914729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:17.914729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:17.914729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:17.914729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:17.914729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:17.914729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:17.914729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:17.914729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:17.914729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:17.914729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:17.914729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:06:17.914729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:17.914729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 427/850] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:06:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:17.916287 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8e952-944b-4477-ba95-1011a16872d2 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:17.940795 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05703bc5-dcad-41f9-babc-7314466526a5 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:17.993988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c7419-6cda-40e4-8aca-9a743282a36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:17.994901 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c7419-6cda-40e4-8aca-9a743282a36e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:17.995423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c7419-6cda-40e4-8aca-9a743282a36e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:17.995852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c7419-6cda-40e4-8aca-9a743282a36e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:18.005038 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c7419-6cda-40e4-8aca-9a743282a36e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:18.006728 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c7419-6cda-40e4-8aca-9a743282a36e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:18.018895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-611c7419-6cda-40e4-8aca-9a743282a36e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:06:18.018895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:18.018895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:18.018895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:18.018895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:18.018895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:18.018895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:18.018895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:18.018895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:18.018895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:18.018895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:18.018895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:18.018895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:06:18.018895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:06:18.018895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:06:18.018895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:18.020050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:18.021221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:18.021221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:18.021221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:06:18.021221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:06:18.021221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:06:18.021221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:06:18.021221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:06:18.021221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:06:18.021221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:06:18.021221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:18.023054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 424/851] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 31 19:06:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:18.023948 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c7419-6cda-40e4-8aca-9a743282a36e None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:18.024469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8e952-944b-4477-ba95-1011a16872d2 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:18.055952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:18.066230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c103b02-d1b3-40f8-9aa3-d4c3c366e2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:18.067005 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c103b02-d1b3-40f8-9aa3-d4c3c366e2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:18.067426 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c103b02-d1b3-40f8-9aa3-d4c3c366e2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:18.067841 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c103b02-d1b3-40f8-9aa3-d4c3c366e2d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:18.080220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c103b02-d1b3-40f8-9aa3-d4c3c366e2d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:18.082579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c103b02-d1b3-40f8-9aa3-d4c3c366e2d7 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:18.101652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-1c103b02-d1b3-40f8-9aa3-d4c3c366e2d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:18.101652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:18.101652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:18.101652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:18.101652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:18.101652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:18.101652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:18.101652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:18.101652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:18.101652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:18.101652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:18.101652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:18.101652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:18.101652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:18.101652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:18.101652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:18.102645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:18.102645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:18.102645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:18.102645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:18.102645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:18.102645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:18.102645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:18.102645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:18.102645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:18.102645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:18.102645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:18.102645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:18.102645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:18.102645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:18.102645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:18.102645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:18.102645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:18.103608 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:18.103608 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:18.103608 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:18.103608 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 428/852] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:18.104283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c103b02-d1b3-40f8-9aa3-d4c3c366e2d7 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:18.112211 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:18.136620 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c7419-6cda-40e4-8aca-9a743282a36e None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:18.176064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ad464-0c63-459b-8515-6294cd8b3184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:18.176675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ad464-0c63-459b-8515-6294cd8b3184 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:18.176960 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ad464-0c63-459b-8515-6294cd8b3184 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:18.177230 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ad464-0c63-459b-8515-6294cd8b3184 None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:18.177862 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ad464-0c63-459b-8515-6294cd8b3184 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:18.178166 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ad464-0c63-459b-8515-6294cd8b3184 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:18.179573 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ad464-0c63-459b-8515-6294cd8b3184 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:18.184123 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 425/853] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:06:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:18.184886 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ad464-0c63-459b-8515-6294cd8b3184 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:18.215104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c103b02-d1b3-40f8-9aa3-d4c3c366e2d7 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:18.248851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac96ab-46b4-4a4f-8711-ac43e5d77b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:18.249563 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac96ab-46b4-4a4f-8711-ac43e5d77b9c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:18.249928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac96ab-46b4-4a4f-8711-ac43e5d77b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:18.250277 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac96ab-46b4-4a4f-8711-ac43e5d77b9c None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:18.251103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac96ab-46b4-4a4f-8711-ac43e5d77b9c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:18.251474 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac96ab-46b4-4a4f-8711-ac43e5d77b9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:18.253043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac96ab-46b4-4a4f-8711-ac43e5d77b9c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:18.272526 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 429/854] 10.222.0.59 () {56 vars in 1149 bytes} [Wed Aug 31 19:06:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:18.274125 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0757e2d-d4c3-4bfa-987c-ed12b3371b14 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:18.274419 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0757e2d-d4c3-4bfa-987c-ed12b3371b14 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:18.274800 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0757e2d-d4c3-4bfa-987c-ed12b3371b14 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:18.276063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 430/855] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:18.276766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0757e2d-d4c3-4bfa-987c-ed12b3371b14 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:18.294613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ad464-0c63-459b-8515-6294cd8b3184 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:18.314884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:18.329745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5d513-abc5-44ab-a974-51770761de00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:18.331101 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5d513-abc5-44ab-a974-51770761de00 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:18.331375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5d513-abc5-44ab-a974-51770761de00 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:18.331646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5d513-abc5-44ab-a974-51770761de00 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:18.332376 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5d513-abc5-44ab-a974-51770761de00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:18.332676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5d513-abc5-44ab-a974-51770761de00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:18.334156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5d513-abc5-44ab-a974-51770761de00 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:18.343665 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:18.347746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 426/856] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:18.352485 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5d513-abc5-44ab-a974-51770761de00 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:18.388400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0757e2d-d4c3-4bfa-987c-ed12b3371b14 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:18.426933 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682bc6e-448a-4eb7-810c-78e55716461f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:18.427833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682bc6e-448a-4eb7-810c-78e55716461f None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:18.428472 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682bc6e-448a-4eb7-810c-78e55716461f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:18.428597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682bc6e-448a-4eb7-810c-78e55716461f None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:18.429509 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682bc6e-448a-4eb7-810c-78e55716461f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:18.429949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682bc6e-448a-4eb7-810c-78e55716461f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:18.432005 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682bc6e-448a-4eb7-810c-78e55716461f None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:18.439025 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 431/857] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:06:18 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:18.440776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749fea4-2420-40e9-894e-0b8bf9ef8f60 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:18.441212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749fea4-2420-40e9-894e-0b8bf9ef8f60 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:18.441655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749fea4-2420-40e9-894e-0b8bf9ef8f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:18.453146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5d513-abc5-44ab-a974-51770761de00 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:18.469554 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-7749fea4-2420-40e9-894e-0b8bf9ef8f60 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:18.473517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7749fea4-2420-40e9-894e-0b8bf9ef8f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:18.506404 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fca61c-2aa6-4853-849b-978c92c968fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:18.507394 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fca61c-2aa6-4853-849b-978c92c968fb None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:18.507848 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fca61c-2aa6-4853-849b-978c92c968fb None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:18.508290 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fca61c-2aa6-4853-849b-978c92c968fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:18.586961 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 432/858] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:18.587135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749fea4-2420-40e9-894e-0b8bf9ef8f60 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:18.606356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:18.612289 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fca61c-2aa6-4853-849b-978c92c968fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:18.612448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fca61c-2aa6-4853-849b-978c92c968fb None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:18.621217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:18.716062 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749fea4-2420-40e9-894e-0b8bf9ef8f60 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:18.733071 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 427/859] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:06:18 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:06:18.738529 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fca61c-2aa6-4853-849b-978c92c968fb None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:18.760296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887de04c-ee5c-497e-9021-b9d8433ce422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:18.761535 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887de04c-ee5c-497e-9021-b9d8433ce422 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:18.762421 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887de04c-ee5c-497e-9021-b9d8433ce422 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:18.762939 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887de04c-ee5c-497e-9021-b9d8433ce422 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:18.773637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887de04c-ee5c-497e-9021-b9d8433ce422 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:18.775950 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887de04c-ee5c-497e-9021-b9d8433ce422 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:18.792798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-887de04c-ee5c-497e-9021-b9d8433ce422 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:18.792798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:18.792798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:18.792798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:18.792798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:18.792798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:18.792798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:18.792798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:18.792798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:18.792798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:18.792798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:18.792798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:18.792798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:06:18.792798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:06:18.792798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:06:18.793771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:06:18.793771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:18.793771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:18.793771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:18.793771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:18.793771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:18.793771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:18.793771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:18.793771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:18.793771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:18.793771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:18.793771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:18.793771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:18.793771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:18.793771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:18.793771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:18.793771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:18.794720 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:06:18.794720 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:06:18.794720 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:06:18.794720 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:06:18.794720 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:06:18.794720 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:06:18.794720 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:18.794720 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:18.802255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 433/860] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:18.804070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887de04c-ee5c-497e-9021-b9d8433ce422 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:18.864264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:18.867461 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fca61c-2aa6-4853-849b-978c92c968fb None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:18.905984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e54ad0-5f21-4e77-bc85-52bd78c0a836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:18.906369 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:18.907374 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e54ad0-5f21-4e77-bc85-52bd78c0a836 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:18.907661 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e54ad0-5f21-4e77-bc85-52bd78c0a836 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:18.907934 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e54ad0-5f21-4e77-bc85-52bd78c0a836 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:18.917646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887de04c-ee5c-497e-9021-b9d8433ce422 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:18.920040 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e54ad0-5f21-4e77-bc85-52bd78c0a836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:18.921545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e54ad0-5f21-4e77-bc85-52bd78c0a836 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:18.937682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-87e54ad0-5f21-4e77-bc85-52bd78c0a836 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:18.937682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:18.937682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:18.937682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:18.937682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:18.937682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:18.937682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:18.937682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:18.937682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:18.937682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:18.937682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:18.937682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:18.937682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:18.937682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:18.937682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:18.937682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:18.938796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:18.938796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:18.938796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:18.938796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:18.938796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:18.938796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:18.938796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:18.938796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:18.938796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:18.938796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:18.938796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:18.938796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:18.938796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:18.938796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:18.938796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:18.938796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:18.938796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:18.940131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:18.940131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:18.940131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:18.943972 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 428/861] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:18.944747 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e54ad0-5f21-4e77-bc85-52bd78c0a836 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:18.959553 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e599843-f8c3-4ba5-86cc-73a424b51590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:18.960176 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e599843-f8c3-4ba5-86cc-73a424b51590 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:18.960457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e599843-f8c3-4ba5-86cc-73a424b51590 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:18.960733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e599843-f8c3-4ba5-86cc-73a424b51590 None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:18.961440 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e599843-f8c3-4ba5-86cc-73a424b51590 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:18.961751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e599843-f8c3-4ba5-86cc-73a424b51590 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:18.963219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e599843-f8c3-4ba5-86cc-73a424b51590 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:18.982827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 434/862] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:06:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:18.987860 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e599843-f8c3-4ba5-86cc-73a424b51590 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:19.061931 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e54ad0-5f21-4e77-bc85-52bd78c0a836 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:19.098976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e599843-f8c3-4ba5-86cc-73a424b51590 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:19.101361 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ae6f8-c23e-49f9-8d21-56b52e673e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:19.102003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ae6f8-c23e-49f9-8d21-56b52e673e3d None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:19.102633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ae6f8-c23e-49f9-8d21-56b52e673e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:19.102633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ae6f8-c23e-49f9-8d21-56b52e673e3d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:19.103238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ae6f8-c23e-49f9-8d21-56b52e673e3d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:19.104551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ae6f8-c23e-49f9-8d21-56b52e673e3d None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:19.114126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-ac4ae6f8-c23e-49f9-8d21-56b52e673e3d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:06:19.114126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:19.114126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:19.114126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:19.114126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:19.114126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:19.114126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:19.114126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:19.114126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:06:19.114126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:19.114126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:19.114126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:19.114126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:19.114126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:19.114126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:19.114126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:19.115363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:19.115363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:19.115363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:19.115363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:19.115363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:19.115363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:19.115363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:19.115363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:19.115363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:19.115363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:19.115363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:19.115363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:19.115363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:19.115363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:19.115363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:19.115363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:19.115363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:19.116281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:19.116281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:19.116281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:19.116281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:19.116281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:19.116281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:19.116281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:19.116281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:19.116281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:19.116281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:19.116281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:19.116281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:19.116281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:06:19.116281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:19.116281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 429/863] 10.222.0.59 () {56 vars in 1132 bytes} [Wed Aug 31 19:06:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:19.117093 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ae6f8-c23e-49f9-8d21-56b52e673e3d None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:19.138095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:19.145155 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef5520-4c2f-4714-9baf-2e860cd2a17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:19.152238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef5520-4c2f-4714-9baf-2e860cd2a17d None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:19.152238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef5520-4c2f-4714-9baf-2e860cd2a17d None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:19.152238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef5520-4c2f-4714-9baf-2e860cd2a17d None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:19.152238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef5520-4c2f-4714-9baf-2e860cd2a17d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:19.152238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef5520-4c2f-4714-9baf-2e860cd2a17d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:19.152238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef5520-4c2f-4714-9baf-2e860cd2a17d None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:19.166014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 435/864] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:19.166014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef5520-4c2f-4714-9baf-2e860cd2a17d None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:19.179641 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:19.237986 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ae6f8-c23e-49f9-8d21-56b52e673e3d None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:19.270377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef5520-4c2f-4714-9baf-2e860cd2a17d None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:19.279116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24412ea5-5c0f-402f-94bc-921335754fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:19.280111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24412ea5-5c0f-402f-94bc-921335754fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:19.280279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24412ea5-5c0f-402f-94bc-921335754fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:19.280656 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24412ea5-5c0f-402f-94bc-921335754fc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:19.281809 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24412ea5-5c0f-402f-94bc-921335754fc3 None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=25ed58369ffd46e8b7fa1880662926e8) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:19.281809 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24412ea5-5c0f-402f-94bc-921335754fc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:19.284093 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24412ea5-5c0f-402f-94bc-921335754fc3 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:19.290621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 430/865] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:06:19 2022] GET /identity/v3/role_assignments?user.id=25ed58369ffd46e8b7fa1880662926e8&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:19.291752 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24412ea5-5c0f-402f-94bc-921335754fc3 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:19.333820 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34b806-f934-4d0e-9871-4243516c9ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:19.335127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34b806-f934-4d0e-9871-4243516c9ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:19.335593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34b806-f934-4d0e-9871-4243516c9ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:19.337061 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34b806-f934-4d0e-9871-4243516c9ac4 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:19.337947 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34b806-f934-4d0e-9871-4243516c9ac4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:19.338437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34b806-f934-4d0e-9871-4243516c9ac4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:19.340065 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34b806-f934-4d0e-9871-4243516c9ac4 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:19.347594 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 436/866] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 31 19:06:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:19.352954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34b806-f934-4d0e-9871-4243516c9ac4 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:19.422652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24412ea5-5c0f-402f-94bc-921335754fc3 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:19.447143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:19.447504 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:19.478156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557860a-7e5c-495c-a946-387c481b7a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:19.479061 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557860a-7e5c-495c-a946-387c481b7a15 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:19.479424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557860a-7e5c-495c-a946-387c481b7a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:19.479690 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557860a-7e5c-495c-a946-387c481b7a15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:19.481990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34b806-f934-4d0e-9871-4243516c9ac4 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:19.489120 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557860a-7e5c-495c-a946-387c481b7a15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:19.489948 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557860a-7e5c-495c-a946-387c481b7a15 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:19.502807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-0557860a-7e5c-495c-a946-387c481b7a15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:19.502807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:19.502807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:19.502807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:19.502807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:19.502807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:19.502807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:19.502807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:19.502807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:19.502807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:19.502807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:19.502807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:19.502807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:06:19.502807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:06:19.502807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:06:19.503847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:06:19.503847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:19.503847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:19.503847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:19.503847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:19.503847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:19.503847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:19.503847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:19.503847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:19.503847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:19.503847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:19.503847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:19.503847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:19.503847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:19.503847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:19.503847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:19.503847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:19.504761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:06:19.504761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:06:19.504761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:06:19.504761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:06:19.504761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:06:19.504761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:06:19.504761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:19.504761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:19.506914 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 431/867] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:19.512022 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557860a-7e5c-495c-a946-387c481b7a15 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:19.534312 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bf2da-44c7-4d72-844f-b1a23ed32613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:19.534993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bf2da-44c7-4d72-844f-b1a23ed32613 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:19.535275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bf2da-44c7-4d72-844f-b1a23ed32613 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:19.535546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bf2da-44c7-4d72-844f-b1a23ed32613 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:19.536775 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bf2da-44c7-4d72-844f-b1a23ed32613 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:19.536775 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bf2da-44c7-4d72-844f-b1a23ed32613 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:19.539371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bf2da-44c7-4d72-844f-b1a23ed32613 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:19.545903 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 437/868] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 31 19:06:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:19.639646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557860a-7e5c-495c-a946-387c481b7a15 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:19.680935 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c827067-c4ff-4892-8ff5-b8894f47cf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:19.681548 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c827067-c4ff-4892-8ff5-b8894f47cf64 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:19.681836 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c827067-c4ff-4892-8ff5-b8894f47cf64 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:19.682113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c827067-c4ff-4892-8ff5-b8894f47cf64 None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:19.682817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c827067-c4ff-4892-8ff5-b8894f47cf64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:19.683152 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c827067-c4ff-4892-8ff5-b8894f47cf64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:19.684606 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c827067-c4ff-4892-8ff5-b8894f47cf64 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:19.695857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:19.700238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 432/869] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:06:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:19.707582 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bf2da-44c7-4d72-844f-b1a23ed32613 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:19.815948 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bf2da-44c7-4d72-844f-b1a23ed32613 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:19.851523 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf0c001-9d3a-4518-ae37-d0fa41fefb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:19.852133 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0c001-9d3a-4518-ae37-d0fa41fefb1f None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:19.852415 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0c001-9d3a-4518-ae37-d0fa41fefb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:19.852754 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0c001-9d3a-4518-ae37-d0fa41fefb1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:19.853543 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0c001-9d3a-4518-ae37-d0fa41fefb1f None admin] RBAC: Adding query filter params (role.id=a9ef0098e982437fb0127827fa2cad38, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=1ee990b1658e49cc8bc2fe15e7dc36d9) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:19.853859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0c001-9d3a-4518-ae37-d0fa41fefb1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:19.855322 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0c001-9d3a-4518-ae37-d0fa41fefb1f None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:19.863196 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 438/870] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:06:19 2022] GET /identity/v3/role_assignments?user.id=1ee990b1658e49cc8bc2fe15e7dc36d9&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=a9ef0098e982437fb0127827fa2cad38 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:20.431847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e65b5-5f07-42b6-84cb-2f08c6adcf86 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:20.432252 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e65b5-5f07-42b6-84cb-2f08c6adcf86 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:20.432437 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e65b5-5f07-42b6-84cb-2f08c6adcf86 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:20.434055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 433/871] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:20.440095 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195af73c-38f7-4379-b224-8bbf8b4a43c9 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:20.440378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195af73c-38f7-4379-b224-8bbf8b4a43c9 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:20.440746 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195af73c-38f7-4379-b224-8bbf8b4a43c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:20.446956 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:20.459946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-195af73c-38f7-4379-b224-8bbf8b4a43c9 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:20.461135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-195af73c-38f7-4379-b224-8bbf8b4a43c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:20.537901 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 439/872] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:20.816391 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e65b5-5f07-42b6-84cb-2f08c6adcf86 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:20.928395 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e65b5-5f07-42b6-84cb-2f08c6adcf86 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:20.963588 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f7c21-7f39-4862-8f32-f81debadd709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:20.964198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f7c21-7f39-4862-8f32-f81debadd709 None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:20.964476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f7c21-7f39-4862-8f32-f81debadd709 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:20.964748 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f7c21-7f39-4862-8f32-f81debadd709 None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:20.965475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f7c21-7f39-4862-8f32-f81debadd709 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:20.966887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f7c21-7f39-4862-8f32-f81debadd709 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:21.077665 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 434/873] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 31 19:06:20 2022] POST /identity/v3/services => generated 251 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:21.420745 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2bd58-09c9-4baa-bf09-3790c6a66802 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:21.420943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2bd58-09c9-4baa-bf09-3790c6a66802 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:21.421216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2bd58-09c9-4baa-bf09-3790c6a66802 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:21.422850 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 440/874] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:21.428744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f28eb-1f76-4f37-8dcd-eaa76ee7c592 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:21.429040 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f28eb-1f76-4f37-8dcd-eaa76ee7c592 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:21.429325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f28eb-1f76-4f37-8dcd-eaa76ee7c592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:21.449143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-1f7f28eb-1f76-4f37-8dcd-eaa76ee7c592 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:21.450346 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-1f7f28eb-1f76-4f37-8dcd-eaa76ee7c592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:21.511061 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:21.530807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 435/875] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:21 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:21.615039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1ab10-349f-4cd7-b5df-7359dde051c5 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:21.616003 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1ab10-349f-4cd7-b5df-7359dde051c5 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:21.616483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1ab10-349f-4cd7-b5df-7359dde051c5 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:21.618459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 441/876] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:21.624259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5ae67-4eea-495a-b61e-ba3d1e58adce None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:21.624791 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5ae67-4eea-495a-b61e-ba3d1e58adce None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:21.625296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5ae67-4eea-495a-b61e-ba3d1e58adce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:21.647349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-60c5ae67-4eea-495a-b61e-ba3d1e58adce None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:21.648871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-60c5ae67-4eea-495a-b61e-ba3d1e58adce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:21.727109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 436/877] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:21 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:21.806146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1ab10-349f-4cd7-b5df-7359dde051c5 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:21.915363 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1ab10-349f-4cd7-b5df-7359dde051c5 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:21.938105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a111a4f-3e47-47d0-a215-e363652daacb None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:21.938609 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a111a4f-3e47-47d0-a215-e363652daacb None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:21.938686 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a111a4f-3e47-47d0-a215-e363652daacb None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:21.940428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 437/878] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:21.945318 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd8fe3-0a96-4ea1-8ef4-879bf4267533 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:21.945599 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd8fe3-0a96-4ea1-8ef4-879bf4267533 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:21.946111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd8fe3-0a96-4ea1-8ef4-879bf4267533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:21.946616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:21.951947 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21888253-0207-4a56-8148-721a37f94d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:21.952595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21888253-0207-4a56-8148-721a37f94d65 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:21.952876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21888253-0207-4a56-8148-721a37f94d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:21.953152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21888253-0207-4a56-8148-721a37f94d65 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:21.953757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21888253-0207-4a56-8148-721a37f94d65 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:21.955680 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21888253-0207-4a56-8148-721a37f94d65 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:21.964866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-21888253-0207-4a56-8148-721a37f94d65 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:06:21.964866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:21.964866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:21.964866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:21.964866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:21.964866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:21.964866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:21.964866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:21.964866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:06:21.964866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:21.964866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:21.964866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:21.964866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:21.964866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:21.964866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:21.964866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:21.964866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:21.969753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:21.969753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:21.969753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:21.969753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:21.969753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:21.969753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:21.969753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:21.969753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:21.969753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:21.969753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:21.969753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:21.969753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:21.969753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:21.969753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:21.969753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:21.969753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:21.971214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:21.971214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:21.971214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:21.971214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:21.971214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:21.971214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:21.971214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:21.971214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:21.971214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:21.971214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:21.971214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:21.971214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:21.971214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:06:21.971214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:21.971993 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-96cd8fe3-0a96-4ea1-8ef4-879bf4267533 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:21.972071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 442/879] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 31 19:06:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:21.972151 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-96cd8fe3-0a96-4ea1-8ef4-879bf4267533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:21.973253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21888253-0207-4a56-8148-721a37f94d65 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:22.041553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 438/880] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:21 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:22.042293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd8fe3-0a96-4ea1-8ef4-879bf4267533 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:22.092208 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:22.102388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21888253-0207-4a56-8148-721a37f94d65 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:22.148245 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f21f73-f085-4f21-b6a3-8dbf87e10316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:22.149241 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f21f73-f085-4f21-b6a3-8dbf87e10316 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:22.149960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f21f73-f085-4f21-b6a3-8dbf87e10316 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:22.150667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f21f73-f085-4f21-b6a3-8dbf87e10316 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:22.151520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f21f73-f085-4f21-b6a3-8dbf87e10316 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:22.152107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f21f73-f085-4f21-b6a3-8dbf87e10316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:22.154249 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f21f73-f085-4f21-b6a3-8dbf87e10316 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:22.155391 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd8fe3-0a96-4ea1-8ef4-879bf4267533 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:22.168169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 443/881] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 31 19:06:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:22.168833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f21f73-f085-4f21-b6a3-8dbf87e10316 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:22.207895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ae666-1676-4b4c-882d-4f15a0b390e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:22.209128 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ae666-1676-4b4c-882d-4f15a0b390e5 None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:22.209630 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ae666-1676-4b4c-882d-4f15a0b390e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:22.211090 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ae666-1676-4b4c-882d-4f15a0b390e5 None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:22.211979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ae666-1676-4b4c-882d-4f15a0b390e5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:22.213558 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ae666-1676-4b4c-882d-4f15a0b390e5 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:22.242583 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:22.295148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f21f73-f085-4f21-b6a3-8dbf87e10316 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:22.308340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 439/882] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 31 19:06:22 2022] POST /identity/v3/services => generated 269 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:22.341064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ae666-1676-4b4c-882d-4f15a0b390e5 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:22.358824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598b7c8-b31e-4586-a570-0858dda12fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:22.359488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598b7c8-b31e-4586-a570-0858dda12fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:22.359770 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598b7c8-b31e-4586-a570-0858dda12fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:22.360043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598b7c8-b31e-4586-a570-0858dda12fc9 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:22.360703 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598b7c8-b31e-4586-a570-0858dda12fc9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:22.361288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598b7c8-b31e-4586-a570-0858dda12fc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:22.364117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598b7c8-b31e-4586-a570-0858dda12fc9 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:22.371619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 444/883] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 31 19:06:22 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:22.479097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ae666-1676-4b4c-882d-4f15a0b390e5 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:22.490699 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:22.525434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bfd65b-5faf-4390-b4e8-180df05af1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:22.526426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfd65b-5faf-4390-b4e8-180df05af1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:22.526904 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfd65b-5faf-4390-b4e8-180df05af1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:22.527374 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfd65b-5faf-4390-b4e8-180df05af1d5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:22.539749 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bfd65b-5faf-4390-b4e8-180df05af1d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:22.545420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-a7bfd65b-5faf-4390-b4e8-180df05af1d5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:06:22.545420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:22.545420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:22.545420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:22.545420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:22.545420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:22.545420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:22.545420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:22.545420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:22.545420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:22.545420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:22.545420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:22.545420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:22.545420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:22.545420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:22.545420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:22.546372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:22.546372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:22.546372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:22.546372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:22.546372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:22.546372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:22.546372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:22.546372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:22.546372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:22.546372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:22.546372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:22.546372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:22.546372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:22.546372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:22.546372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:22.546372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:22.547293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:22.547293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:22.547293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:22.547293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:22.547293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:22.547293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:22.547293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:22.547293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:22.547293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:22.547293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:22.547293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:06:22.547293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:22.547293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 440/884] 10.222.0.59 () {56 vars in 1120 bytes} [Wed Aug 31 19:06:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:22.554999 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598b7c8-b31e-4586-a570-0858dda12fc9 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:22.670156 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598b7c8-b31e-4586-a570-0858dda12fc9 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:22.692919 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:22.708832 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81288ac9-5ed8-4669-b5b9-7265b9a2d451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:22.709492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81288ac9-5ed8-4669-b5b9-7265b9a2d451 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:22.709774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81288ac9-5ed8-4669-b5b9-7265b9a2d451 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:22.710047 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81288ac9-5ed8-4669-b5b9-7265b9a2d451 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:22.710717 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81288ac9-5ed8-4669-b5b9-7265b9a2d451 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:22.711053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81288ac9-5ed8-4669-b5b9-7265b9a2d451 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:22.712442 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81288ac9-5ed8-4669-b5b9-7265b9a2d451 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:22.722587 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 445/885] 10.222.0.59 () {56 vars in 1115 bytes} [Wed Aug 31 19:06:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:22.728641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bfd65b-5faf-4390-b4e8-180df05af1d5 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:22.839765 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bfd65b-5faf-4390-b4e8-180df05af1d5 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:22.879451 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd34138-a1e9-4503-88e3-4f2448e0188a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:22.880332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd34138-a1e9-4503-88e3-4f2448e0188a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:22.880760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd34138-a1e9-4503-88e3-4f2448e0188a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:22.881205 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd34138-a1e9-4503-88e3-4f2448e0188a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:22.893738 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd34138-a1e9-4503-88e3-4f2448e0188a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:22.895450 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd34138-a1e9-4503-88e3-4f2448e0188a None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:22.911684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-7bd34138-a1e9-4503-88e3-4f2448e0188a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:22.911684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:22.911684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:22.911684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:22.911684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:22.911684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:22.911684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:22.911684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:22.911684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:22.911684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:22.911684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:22.911684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:22.911684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:22.911684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:22.911684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:22.911684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:22.913019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:22.913019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:22.913019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:22.913019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:22.913019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:22.913019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:22.913019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:22.913019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:22.913019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:22.913019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:22.913019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:22.913019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:22.913019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:22.913019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:22.913019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:22.913019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:22.913019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:22.914150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:22.914150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:22.914150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:22.915978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 441/886] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:22.921703 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81288ac9-5ed8-4669-b5b9-7265b9a2d451 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:23.028419 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81288ac9-5ed8-4669-b5b9-7265b9a2d451 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:23.065009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c87e1-ccae-451a-a046-c2a4b23be955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:23.065626 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c87e1-ccae-451a-a046-c2a4b23be955 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:23.065903 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c87e1-ccae-451a-a046-c2a4b23be955 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:23.066178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c87e1-ccae-451a-a046-c2a4b23be955 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:23.066821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c87e1-ccae-451a-a046-c2a4b23be955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:23.067150 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c87e1-ccae-451a-a046-c2a4b23be955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:23.068518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c87e1-ccae-451a-a046-c2a4b23be955 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:23.081768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 446/887] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:23.087977 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd34138-a1e9-4503-88e3-4f2448e0188a None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:23.096727 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004e6be-ac6d-4dc9-9cf0-c870e6baa156 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:23.097081 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004e6be-ac6d-4dc9-9cf0-c870e6baa156 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:23.097364 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004e6be-ac6d-4dc9-9cf0-c870e6baa156 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:23.098998 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 447/888] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:23.104026 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06df23a-2023-437e-8653-f266e5fd31f7 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:23.104313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06df23a-2023-437e-8653-f266e5fd31f7 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:23.104596 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06df23a-2023-437e-8653-f266e5fd31f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:23.111276 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:23.124268 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d06df23a-2023-437e-8653-f266e5fd31f7 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:23.125470 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d06df23a-2023-437e-8653-f266e5fd31f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:23.164870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:23.206986 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 448/889] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:23.218813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd34138-a1e9-4503-88e3-4f2448e0188a None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:23.255125 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365280e-8e1f-464f-a428-02b559c572fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:23.256006 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365280e-8e1f-464f-a428-02b559c572fe None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:23.256429 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365280e-8e1f-464f-a428-02b559c572fe None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:23.256841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365280e-8e1f-464f-a428-02b559c572fe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:23.264883 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365280e-8e1f-464f-a428-02b559c572fe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:23.266495 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365280e-8e1f-464f-a428-02b559c572fe None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:23.278325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-b365280e-8e1f-464f-a428-02b559c572fe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:06:23.278325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:23.278325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:23.278325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:23.278325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:23.278325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:23.278325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:23.278325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:23.278325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:23.278325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:23.278325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:23.278325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:23.278325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:06:23.278325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:06:23.278325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:06:23.278325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:23.279522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:23.280701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:23.280701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:23.280701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:06:23.280701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:06:23.280701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:06:23.280701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:06:23.280701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:06:23.280701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:06:23.280701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:06:23.280701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:23.280701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 442/890] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 31 19:06:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:23.286134 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06df23a-2023-437e-8653-f266e5fd31f7 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:23.393372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06df23a-2023-437e-8653-f266e5fd31f7 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:23.429198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f69c3-1a34-49e1-8b92-cc3fc3dabb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:23.429814 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f69c3-1a34-49e1-8b92-cc3fc3dabb71 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:23.430093 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f69c3-1a34-49e1-8b92-cc3fc3dabb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:23.430368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f69c3-1a34-49e1-8b92-cc3fc3dabb71 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:23.431134 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f69c3-1a34-49e1-8b92-cc3fc3dabb71 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:23.431431 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f69c3-1a34-49e1-8b92-cc3fc3dabb71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:23.432861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f69c3-1a34-49e1-8b92-cc3fc3dabb71 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:23.452645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 449/891] 10.222.0.59 () {56 vars in 1149 bytes} [Wed Aug 31 19:06:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:23.458955 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365280e-8e1f-464f-a428-02b559c572fe None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:23.481075 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f69c3-1a34-49e1-8b92-cc3fc3dabb71 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:23.514643 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:23.566326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:23.598966 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365280e-8e1f-464f-a428-02b559c572fe None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:23.600566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f69c3-1a34-49e1-8b92-cc3fc3dabb71 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:23.638755 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa986e-7f31-4cf9-a24a-285e328ca777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:23.639533 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa986e-7f31-4cf9-a24a-285e328ca777 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:23.639921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa986e-7f31-4cf9-a24a-285e328ca777 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:23.640307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa986e-7f31-4cf9-a24a-285e328ca777 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:23.640962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa986e-7f31-4cf9-a24a-285e328ca777 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:23.642037 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26c1ef-7fd9-48c4-aeb2-a67a9fa4dfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:23.643110 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa986e-7f31-4cf9-a24a-285e328ca777 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:23.643519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26c1ef-7fd9-48c4-aeb2-a67a9fa4dfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:23.643938 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26c1ef-7fd9-48c4-aeb2-a67a9fa4dfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:23.644212 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26c1ef-7fd9-48c4-aeb2-a67a9fa4dfc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:23.652710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-68aa986e-7f31-4cf9-a24a-285e328ca777 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:06:23.652710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:23.652710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:23.652710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:23.652710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:23.652710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:23.652710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:23.652710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:23.652710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:06:23.652710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:23.652710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:23.652710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:23.652710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:23.652710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:23.652710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:23.652710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:23.652710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:23.655012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:23.655012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:23.655012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:23.655012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:23.655012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:23.655012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:23.655012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:23.655012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:23.655012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:23.655012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:23.655012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:23.655012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:23.655012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:23.655012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:23.655012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:23.655012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:23.657050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:23.657050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:23.657050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:23.657050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:23.657050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:23.657050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:23.657050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:23.657050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:23.657050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:23.657050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:23.657050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:23.657050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:23.657050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:06:23.657050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:23.667108 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 450/892] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:06:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:23.668365 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa986e-7f31-4cf9-a24a-285e328ca777 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:23.719780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26c1ef-7fd9-48c4-aeb2-a67a9fa4dfc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:23.722338 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26c1ef-7fd9-48c4-aeb2-a67a9fa4dfc6 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:23.770200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:23.777657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa986e-7f31-4cf9-a24a-285e328ca777 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:23.805766 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:23.813023 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d27c6-8a77-451f-b425-a6d0e3334c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:23.813924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d27c6-8a77-451f-b425-a6d0e3334c22 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:23.815101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d27c6-8a77-451f-b425-a6d0e3334c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:23.815587 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d27c6-8a77-451f-b425-a6d0e3334c22 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:23.816453 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d27c6-8a77-451f-b425-a6d0e3334c22 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:23.817101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d27c6-8a77-451f-b425-a6d0e3334c22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:23.819710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d27c6-8a77-451f-b425-a6d0e3334c22 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:23.825321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 451/893] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:06:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:23.831848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d27c6-8a77-451f-b425-a6d0e3334c22 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:23.834300 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 443/894] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:06:23 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:06:23.839861 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26c1ef-7fd9-48c4-aeb2-a67a9fa4dfc6 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:23.945511 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26c1ef-7fd9-48c4-aeb2-a67a9fa4dfc6 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:23.964658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d27c6-8a77-451f-b425-a6d0e3334c22 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:23.988160 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793415c1-957e-4fef-acd1-da17d02d9fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:23.989708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793415c1-957e-4fef-acd1-da17d02d9fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:23.990464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793415c1-957e-4fef-acd1-da17d02d9fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:23.990988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793415c1-957e-4fef-acd1-da17d02d9fc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.003967 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7e267-d78c-4928-96f4-3e81b40ce58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:24.004649 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7e267-d78c-4928-96f4-3e81b40ce58f None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.004916 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7e267-d78c-4928-96f4-3e81b40ce58f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.005244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7e267-d78c-4928-96f4-3e81b40ce58f None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.005896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7e267-d78c-4928-96f4-3e81b40ce58f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:24.006203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7e267-d78c-4928-96f4-3e81b40ce58f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:24.006474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793415c1-957e-4fef-acd1-da17d02d9fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:24.007682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7e267-d78c-4928-96f4-3e81b40ce58f None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:24.008365 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793415c1-957e-4fef-acd1-da17d02d9fc0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:24.018228 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 452/895] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:06:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:24.019403 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7e267-d78c-4928-96f4-3e81b40ce58f None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:24.027145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-793415c1-957e-4fef-acd1-da17d02d9fc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:24.027145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:24.027145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:24.027145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:24.027145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:24.027145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:24.027145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:24.027145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:24.027145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:24.027145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:24.027145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:24.027145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:24.027145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:24.027145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:24.027145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:24.027145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:24.028179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:24.028179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:24.028179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:24.028179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:24.028179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:24.028179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:24.028179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:24.028179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:24.028179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:24.028179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:24.028179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:24.028179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:24.028179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:24.028179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:24.028179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:24.028179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:24.028179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:24.029156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:24.029156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:24.029156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:24.031606 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 444/896] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:24.037703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793415c1-957e-4fef-acd1-da17d02d9fc0 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:24.050535 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:24.079946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:24.138789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7e267-d78c-4928-96f4-3e81b40ce58f None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:24.174908 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793415c1-957e-4fef-acd1-da17d02d9fc0 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:24.177111 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9c344-73cf-4ad5-a2bf-83f7fddd41df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:24.177853 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9c344-73cf-4ad5-a2bf-83f7fddd41df None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.178238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9c344-73cf-4ad5-a2bf-83f7fddd41df None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.179302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9c344-73cf-4ad5-a2bf-83f7fddd41df None admin] PATH_INFO: `/v3/endpoints` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.179497 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9c344-73cf-4ad5-a2bf-83f7fddd41df None admin] RBAC: Adding query filter params (interface=public, service_id=17163a659da6429fb7111eecc7b4bbb4, region_id=RegionOne) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:24.179918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9c344-73cf-4ad5-a2bf-83f7fddd41df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:24.181395 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9c344-73cf-4ad5-a2bf-83f7fddd41df None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:24.192323 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 453/897] 10.222.0.59 () {56 vars in 1251 bytes} [Wed Aug 31 19:06:24 2022] GET /identity/v3/endpoints?service_id=17163a659da6429fb7111eecc7b4bbb4&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:24.193343 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9c344-73cf-4ad5-a2bf-83f7fddd41df None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:24.223259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd039b-f9b8-43e3-bf9b-fa67e4530417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:24.224394 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd039b-f9b8-43e3-bf9b-fa67e4530417 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.224877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd039b-f9b8-43e3-bf9b-fa67e4530417 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.225518 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd039b-f9b8-43e3-bf9b-fa67e4530417 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.226721 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd039b-f9b8-43e3-bf9b-fa67e4530417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:24.227468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd039b-f9b8-43e3-bf9b-fa67e4530417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:24.229659 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd039b-f9b8-43e3-bf9b-fa67e4530417 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:24.247684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 445/898] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:24.254682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd039b-f9b8-43e3-bf9b-fa67e4530417 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:24.307241 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:24.328031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9c344-73cf-4ad5-a2bf-83f7fddd41df None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:24.364804 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:24.375038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb90d5-877e-4ddf-9afc-f73cf7ae01b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:24.375654 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb90d5-877e-4ddf-9afc-f73cf7ae01b4 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.375934 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb90d5-877e-4ddf-9afc-f73cf7ae01b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.376151 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd039b-f9b8-43e3-bf9b-fa67e4530417 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:24.376284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb90d5-877e-4ddf-9afc-f73cf7ae01b4 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.377729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb90d5-877e-4ddf-9afc-f73cf7ae01b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:24.379301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb90d5-877e-4ddf-9afc-f73cf7ae01b4 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:24.390908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 454/899] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 31 19:06:24 2022] GET /identity/v3/services => generated 1268 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:06:24.433112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d08ae3-9c0f-4f12-86c1-dbbf6edde339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:24.433771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d08ae3-9c0f-4f12-86c1-dbbf6edde339 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.434052 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d08ae3-9c0f-4f12-86c1-dbbf6edde339 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.434327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d08ae3-9c0f-4f12-86c1-dbbf6edde339 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.443269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d08ae3-9c0f-4f12-86c1-dbbf6edde339 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:24.444773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d08ae3-9c0f-4f12-86c1-dbbf6edde339 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:24.454474 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a8a62-0dae-4a4e-a773-dba0ffa94dcd None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.455715 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a8a62-0dae-4a4e-a773-dba0ffa94dcd None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.456189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a8a62-0dae-4a4e-a773-dba0ffa94dcd None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.458077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 455/900] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:24.458863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-33d08ae3-9c0f-4f12-86c1-dbbf6edde339 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:24.458863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:24.458863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:24.458863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:24.458863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:24.458863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:24.458863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:24.458863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:24.458863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:24.458863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:24.458863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:24.458863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:24.458863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:06:24.458863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:06:24.458863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:06:24.459772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:06:24.459772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:24.459772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:24.459772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:24.459772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:24.459772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:24.459772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:24.459772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:24.459772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:24.459772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:24.459772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:24.459772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:24.459772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:24.459772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:24.459772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:24.459772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:24.459772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:24.460671 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:06:24.460671 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:06:24.460671 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:06:24.460671 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:06:24.460671 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:06:24.460671 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:06:24.460671 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:24.460671 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:24.460671 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 446/901] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:24.461822 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ec73c-40f9-404f-aa3c-40257654a09b None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.461928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ec73c-40f9-404f-aa3c-40257654a09b None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.462135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ec73c-40f9-404f-aa3c-40257654a09b None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.463665 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210656b-e40f-49a8-a85e-b2447e3183c1 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.464168 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210656b-e40f-49a8-a85e-b2447e3183c1 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.464652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210656b-e40f-49a8-a85e-b2447e3183c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.466978 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 456/902] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:24.468309 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ec73c-40f9-404f-aa3c-40257654a09b None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:24.488097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-0210656b-e40f-49a8-a85e-b2447e3183c1 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:24.489228 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-0210656b-e40f-49a8-a85e-b2447e3183c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:24.574944 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 447/903] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:24.576424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aa7a7-3d7f-4c74-8978-cb5fc3cf7f64 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.576561 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aa7a7-3d7f-4c74-8978-cb5fc3cf7f64 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.577028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aa7a7-3d7f-4c74-8978-cb5fc3cf7f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.594906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ec73c-40f9-404f-aa3c-40257654a09b None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:24.609437 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-596aa7a7-3d7f-4c74-8978-cb5fc3cf7f64 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:24.611347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-596aa7a7-3d7f-4c74-8978-cb5fc3cf7f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:24.643732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63afc4-d8d5-4702-84d0-be1ecd13dd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:24.644368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63afc4-d8d5-4702-84d0-be1ecd13dd37 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.644913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63afc4-d8d5-4702-84d0-be1ecd13dd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.645195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63afc4-d8d5-4702-84d0-be1ecd13dd37 None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.646347 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63afc4-d8d5-4702-84d0-be1ecd13dd37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:24.646956 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63afc4-d8d5-4702-84d0-be1ecd13dd37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:24.648686 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63afc4-d8d5-4702-84d0-be1ecd13dd37 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:24.664955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 457/904] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:06:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:24.671477 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63afc4-d8d5-4702-84d0-be1ecd13dd37 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:24.688469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:24.699662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 448/905] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:24.791210 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63afc4-d8d5-4702-84d0-be1ecd13dd37 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:24.828868 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969825b-6925-4870-94fd-ae2f52eb6c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:24.829484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969825b-6925-4870-94fd-ae2f52eb6c39 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.829925 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969825b-6925-4870-94fd-ae2f52eb6c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.830381 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969825b-6925-4870-94fd-ae2f52eb6c39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.831216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969825b-6925-4870-94fd-ae2f52eb6c39 None admin] RBAC: Adding query filter params (role.id=e22b60b1c78d45b29c81bcce9064de49, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=1ee990b1658e49cc8bc2fe15e7dc36d9) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:24.831535 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969825b-6925-4870-94fd-ae2f52eb6c39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:24.833030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969825b-6925-4870-94fd-ae2f52eb6c39 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:24.839241 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 458/906] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:06:24 2022] GET /identity/v3/role_assignments?user.id=1ee990b1658e49cc8bc2fe15e7dc36d9&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=e22b60b1c78d45b29c81bcce9064de49 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:24.849719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aa7a7-3d7f-4c74-8978-cb5fc3cf7f64 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:24.864421 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:24.972441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aa7a7-3d7f-4c74-8978-cb5fc3cf7f64 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:24.983762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969825b-6925-4870-94fd-ae2f52eb6c39 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:25.012430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507613f2-cfec-41a1-ad48-dc71ea12f973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:25.013077 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507613f2-cfec-41a1-ad48-dc71ea12f973 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:25.013538 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507613f2-cfec-41a1-ad48-dc71ea12f973 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:25.013880 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507613f2-cfec-41a1-ad48-dc71ea12f973 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:25.014478 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507613f2-cfec-41a1-ad48-dc71ea12f973 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:25.015890 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507613f2-cfec-41a1-ad48-dc71ea12f973 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:25.027652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-507613f2-cfec-41a1-ad48-dc71ea12f973 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:06:25.027652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:25.027652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:25.027652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:25.027652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:25.027652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:25.027652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:25.027652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:25.027652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:06:25.027652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:25.027652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:25.027652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:25.027652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:25.027652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:25.027652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:25.027652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:25.028646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:25.028646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:25.028646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:25.028646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:25.028646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:25.028646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:25.028646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:25.028646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:25.028646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:25.028646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:25.028646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:25.028646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:25.028646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:25.028646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:25.028646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:25.028646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:25.028646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:25.029565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:25.029565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:25.029565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:25.029565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:25.029565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:25.029565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:25.029565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:25.029565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:25.029565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:25.029565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:25.029565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:25.029565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:25.029565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:06:25.029565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:25.029565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 449/907] 10.222.0.59 () {56 vars in 1132 bytes} [Wed Aug 31 19:06:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:25.034552 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507613f2-cfec-41a1-ad48-dc71ea12f973 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:25.093585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:25.121371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969825b-6925-4870-94fd-ae2f52eb6c39 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:25.163441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a1d64-0fd5-49f4-8e67-82b2589f6c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:25.164449 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a1d64-0fd5-49f4-8e67-82b2589f6c29 None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:25.165768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:25.166021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a1d64-0fd5-49f4-8e67-82b2589f6c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:25.166562 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a1d64-0fd5-49f4-8e67-82b2589f6c29 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:25.167493 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a1d64-0fd5-49f4-8e67-82b2589f6c29 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:25.169951 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a1d64-0fd5-49f4-8e67-82b2589f6c29 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:25.192953 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507613f2-cfec-41a1-ad48-dc71ea12f973 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:25.229608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b79b3d-49f3-4e61-af7d-4dee4bad523e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:25.230214 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b79b3d-49f3-4e61-af7d-4dee4bad523e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:25.230489 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b79b3d-49f3-4e61-af7d-4dee4bad523e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:25.230759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b79b3d-49f3-4e61-af7d-4dee4bad523e None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:25.231407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b79b3d-49f3-4e61-af7d-4dee4bad523e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:25.231715 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b79b3d-49f3-4e61-af7d-4dee4bad523e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:25.233072 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b79b3d-49f3-4e61-af7d-4dee4bad523e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:25.238900 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 450/908] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 31 19:06:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:25.244382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b79b3d-49f3-4e61-af7d-4dee4bad523e None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:25.251027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 459/909] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 31 19:06:24 2022] POST /identity/v3/services => generated 241 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:25.372912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b79b3d-49f3-4e61-af7d-4dee4bad523e None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:25.425558 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555fa85-44a7-475d-9eb6-fcdc28f9b999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:25.426167 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555fa85-44a7-475d-9eb6-fcdc28f9b999 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:25.426443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555fa85-44a7-475d-9eb6-fcdc28f9b999 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:25.426711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555fa85-44a7-475d-9eb6-fcdc28f9b999 None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:25.427388 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555fa85-44a7-475d-9eb6-fcdc28f9b999 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:25.427692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555fa85-44a7-475d-9eb6-fcdc28f9b999 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:25.429079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555fa85-44a7-475d-9eb6-fcdc28f9b999 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:25.430618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:25.438587 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 451/910] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 31 19:06:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:25.440647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a1d64-0fd5-49f4-8e67-82b2589f6c29 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:25.554571 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a1d64-0fd5-49f4-8e67-82b2589f6c29 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:25.593652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f74774-08fd-4669-86d0-2bf1776b6c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:25.594678 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f74774-08fd-4669-86d0-2bf1776b6c7c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:25.595269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f74774-08fd-4669-86d0-2bf1776b6c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:25.595749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f74774-08fd-4669-86d0-2bf1776b6c7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:25.596659 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f74774-08fd-4669-86d0-2bf1776b6c7c None admin] RBAC: Adding query filter params (interface=public, service_id=bd30714f31344bb4831d2180c3bed98f, region_id=RegionOne) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:25.597169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f74774-08fd-4669-86d0-2bf1776b6c7c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:25.598769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f74774-08fd-4669-86d0-2bf1776b6c7c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:25.600617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:25.609925 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 460/911] 10.222.0.59 () {56 vars in 1251 bytes} [Wed Aug 31 19:06:25 2022] GET /identity/v3/endpoints?service_id=bd30714f31344bb4831d2180c3bed98f&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:25.611838 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555fa85-44a7-475d-9eb6-fcdc28f9b999 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:25.728504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555fa85-44a7-475d-9eb6-fcdc28f9b999 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:25.766267 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356a939-182b-48e7-9c6c-45e95c7f469d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:25.766901 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356a939-182b-48e7-9c6c-45e95c7f469d None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:25.767511 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356a939-182b-48e7-9c6c-45e95c7f469d None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:25.767952 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356a939-182b-48e7-9c6c-45e95c7f469d None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:25.768580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356a939-182b-48e7-9c6c-45e95c7f469d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:25.769966 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356a939-182b-48e7-9c6c-45e95c7f469d None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:25.776848 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 452/912] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 31 19:06:25 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:06:26.463729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcad84-010b-4b72-9afc-ec85b11723c2 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:26.464249 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcad84-010b-4b72-9afc-ec85b11723c2 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:26.464249 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcad84-010b-4b72-9afc-ec85b11723c2 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:26.465990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 461/913] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:26.472304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bafbf-c1da-4f9c-aa90-02d4934bd594 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:26.472849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bafbf-c1da-4f9c-aa90-02d4934bd594 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:26.472849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bafbf-c1da-4f9c-aa90-02d4934bd594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:26.493571 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-351bafbf-c1da-4f9c-aa90-02d4934bd594 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:26.494783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-351bafbf-c1da-4f9c-aa90-02d4934bd594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:26.558722 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:26.571301 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 453/914] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:26.849762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcad84-010b-4b72-9afc-ec85b11723c2 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:26.873668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f22782-056e-4883-9f29-254e5f188c53 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:26.873994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f22782-056e-4883-9f29-254e5f188c53 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:26.874211 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f22782-056e-4883-9f29-254e5f188c53 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:26.875834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 454/915] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:26.881340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc57594-1892-43de-9b84-797b9f746640 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:26.881668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc57594-1892-43de-9b84-797b9f746640 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:26.881879 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc57594-1892-43de-9b84-797b9f746640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:26.902953 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-2dc57594-1892-43de-9b84-797b9f746640 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:26.904316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-2dc57594-1892-43de-9b84-797b9f746640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:26.973572 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcad84-010b-4b72-9afc-ec85b11723c2 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:26.985131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 455/916] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:27.014288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4441e-bba3-41a8-8819-353b3c294d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:27.014959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4441e-bba3-41a8-8819-353b3c294d68 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:27.015295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4441e-bba3-41a8-8819-353b3c294d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:27.015572 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4441e-bba3-41a8-8819-353b3c294d68 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:27.016163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4441e-bba3-41a8-8819-353b3c294d68 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:27.017759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4441e-bba3-41a8-8819-353b3c294d68 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:27.027584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-baf4441e-bba3-41a8-8819-353b3c294d68 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:06:27.027584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:27.027584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:27.027584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:27.027584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:27.027584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:27.027584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:27.027584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:27.027584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:06:27.027584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:27.027584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:27.027584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:27.027584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:27.027584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:27.027584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:27.027584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:27.027584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:27.028704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:27.028704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:27.028704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:27.028704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:27.028704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:27.028704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:27.028704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:27.028704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:27.028704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:27.028704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:27.028704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:27.028704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:27.028704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:27.028704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:27.028704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:27.028704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:27.029748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:27.029748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:27.029748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:27.029748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:27.029748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:27.029748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:27.029748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:27.029748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:27.029748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:27.029748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:27.029748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:27.029748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:27.029748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:06:27.029748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:27.029748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 462/917] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:06:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:27.035695 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc57594-1892-43de-9b84-797b9f746640 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:27.147296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc57594-1892-43de-9b84-797b9f746640 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:27.173030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:27.186252 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218a778-ce0a-4e75-9416-bb843ad53997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:27.186890 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218a778-ce0a-4e75-9416-bb843ad53997 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:27.190283 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218a778-ce0a-4e75-9416-bb843ad53997 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:27.190283 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218a778-ce0a-4e75-9416-bb843ad53997 None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:27.190283 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6218a778-ce0a-4e75-9416-bb843ad53997 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:27.190283 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6218a778-ce0a-4e75-9416-bb843ad53997 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:27.190283 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6218a778-ce0a-4e75-9416-bb843ad53997 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:27.195980 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 456/918] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:06:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:27.201799 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4441e-bba3-41a8-8819-353b3c294d68 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:27.281185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:27.304935 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218a778-ce0a-4e75-9416-bb843ad53997 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:27.339578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4441e-bba3-41a8-8819-353b3c294d68 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:27.387256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eae9d0-8899-495d-8b0d-9e06300786f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:27.387877 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eae9d0-8899-495d-8b0d-9e06300786f0 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:27.388159 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eae9d0-8899-495d-8b0d-9e06300786f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:27.388428 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eae9d0-8899-495d-8b0d-9e06300786f0 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:27.389072 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eae9d0-8899-495d-8b0d-9e06300786f0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:27.389581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eae9d0-8899-495d-8b0d-9e06300786f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:27.391051 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eae9d0-8899-495d-8b0d-9e06300786f0 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:27.396809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 463/919] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 31 19:06:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:27.398077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577b17b-94ff-4236-a7e8-d2476284c56a None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:27.398309 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577b17b-94ff-4236-a7e8-d2476284c56a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:27.398587 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577b17b-94ff-4236-a7e8-d2476284c56a None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:27.400175 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 464/920] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:27.402297 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c577b17b-94ff-4236-a7e8-d2476284c56a None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:27.452760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218a778-ce0a-4e75-9416-bb843ad53997 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:27.491630 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458e42d-013a-4f59-8db5-e80bcbd1ae64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:27.492298 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458e42d-013a-4f59-8db5-e80bcbd1ae64 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:27.492589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458e42d-013a-4f59-8db5-e80bcbd1ae64 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:27.492814 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458e42d-013a-4f59-8db5-e80bcbd1ae64 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:27.505151 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458e42d-013a-4f59-8db5-e80bcbd1ae64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:27.509936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-3458e42d-013a-4f59-8db5-e80bcbd1ae64 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:06:27.509936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:27.509936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:27.509936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:27.509936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:27.509936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:27.509936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:27.509936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:27.509936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:27.509936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:27.509936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:27.509936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:27.509936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:27.509936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:27.509936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:27.509936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:27.511204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:27.511204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:27.511204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:27.511204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:27.511204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:27.511204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:27.511204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:27.511204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:27.511204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:27.511204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:27.511204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:27.511204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:27.511204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:27.511204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:27.511204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:27.511204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:27.512098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:27.512098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:27.512098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:27.512098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:27.512098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:27.512098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:27.512098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:27.512098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:27.512098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:27.512098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:27.512098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:06:27.512098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:27.515196 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 457/921] 10.222.0.59 () {56 vars in 1120 bytes} [Wed Aug 31 19:06:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:27.518170 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b7748-b1de-4354-9c62-fe3cc03e6fee None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:27.518682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b7748-b1de-4354-9c62-fe3cc03e6fee None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:27.519191 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b7748-b1de-4354-9c62-fe3cc03e6fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:27.534721 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c577b17b-94ff-4236-a7e8-d2476284c56a None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:27.551364 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-2f6b7748-b1de-4354-9c62-fe3cc03e6fee None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:27.554573 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-2f6b7748-b1de-4354-9c62-fe3cc03e6fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:27.566007 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:27.575457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bacdc-9c22-429f-922b-48879242d4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:27.576076 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bacdc-9c22-429f-922b-48879242d4a5 None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:27.585134 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bacdc-9c22-429f-922b-48879242d4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:27.585134 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bacdc-9c22-429f-922b-48879242d4a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:27.585134 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35bacdc-9c22-429f-922b-48879242d4a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:27.585134 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35bacdc-9c22-429f-922b-48879242d4a5 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:27.601835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:27.628741 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 458/922] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:27.629638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b7748-b1de-4354-9c62-fe3cc03e6fee None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:27.700503 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 465/923] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:06:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:27.763199 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00ec3c-ef67-4918-84fe-b5dc07948d0b None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:27.763558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00ec3c-ef67-4918-84fe-b5dc07948d0b None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:27.763764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00ec3c-ef67-4918-84fe-b5dc07948d0b None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:27.768990 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b7748-b1de-4354-9c62-fe3cc03e6fee None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:27.771535 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 466/924] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:27.773359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac77a8c-30b0-4e1d-abe4-91efd39b2b4c None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:27.773646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac77a8c-30b0-4e1d-abe4-91efd39b2b4c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:27.773924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac77a8c-30b0-4e1d-abe4-91efd39b2b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:27.795043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-eac77a8c-30b0-4e1d-abe4-91efd39b2b4c None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:27.796432 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-eac77a8c-30b0-4e1d-abe4-91efd39b2b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:27.821906 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae403a9-166d-41ae-b47e-10f9bcc20730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:27.823431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae403a9-166d-41ae-b47e-10f9bcc20730 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:27.824098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae403a9-166d-41ae-b47e-10f9bcc20730 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:27.824614 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae403a9-166d-41ae-b47e-10f9bcc20730 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:27.826251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae403a9-166d-41ae-b47e-10f9bcc20730 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:27.826803 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae403a9-166d-41ae-b47e-10f9bcc20730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:27.829461 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae403a9-166d-41ae-b47e-10f9bcc20730 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:27.853991 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 459/925] 10.222.0.59 () {56 vars in 1115 bytes} [Wed Aug 31 19:06:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:27.854677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae403a9-166d-41ae-b47e-10f9bcc20730 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:27.899022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 467/926] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:27.904720 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:27.912846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac77a8c-30b0-4e1d-abe4-91efd39b2b4c None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:27.989448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae403a9-166d-41ae-b47e-10f9bcc20730 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:28.028593 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffe54a-1e1a-4697-a419-86c1e4e37574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:28.029267 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffe54a-1e1a-4697-a419-86c1e4e37574 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:28.029545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffe54a-1e1a-4697-a419-86c1e4e37574 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:28.029818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffe54a-1e1a-4697-a419-86c1e4e37574 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:28.042049 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffe54a-1e1a-4697-a419-86c1e4e37574 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:28.043296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffe54a-1e1a-4697-a419-86c1e4e37574 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:28.057149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac77a8c-30b0-4e1d-abe4-91efd39b2b4c None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:28.061762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-45ffe54a-1e1a-4697-a419-86c1e4e37574 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:28.061762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:28.061762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:28.061762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:28.061762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:28.061762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:28.061762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:28.061762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:28.061762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:28.061762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:28.061762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:28.061762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:28.061762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:28.061762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:28.061762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:28.061762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:28.062711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:28.062711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:28.062711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:28.062711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:28.062711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:28.062711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:28.062711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:28.062711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:28.062711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:28.062711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:28.062711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:28.062711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:28.062711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:28.062711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:28.062711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:28.062711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:28.062711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:28.063667 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:28.063667 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:28.063667 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:28.074470 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 460/927] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:28.076158 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffe54a-1e1a-4697-a419-86c1e4e37574 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:28.084755 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:28.118847 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a56011-33d8-4b5f-91eb-a42c884ba6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:28.120446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a56011-33d8-4b5f-91eb-a42c884ba6fc None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:28.121270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a56011-33d8-4b5f-91eb-a42c884ba6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:28.122081 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a56011-33d8-4b5f-91eb-a42c884ba6fc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:28.123649 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a56011-33d8-4b5f-91eb-a42c884ba6fc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:28.126246 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a56011-33d8-4b5f-91eb-a42c884ba6fc None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:28.145684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-07a56011-33d8-4b5f-91eb-a42c884ba6fc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:06:28.145684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:28.145684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:28.145684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:28.145684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:28.145684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:28.145684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:28.145684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:28.145684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:06:28.145684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:28.145684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:28.145684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:28.145684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:28.145684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:28.145684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:28.145684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:28.145684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:28.146776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:28.146776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:28.146776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:28.146776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:28.146776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:28.146776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:28.146776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:28.146776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:28.146776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:28.146776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:28.146776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:28.146776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:28.146776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:28.146776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:28.146776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:28.146776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:28.147689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:28.147689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:28.147689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:28.147689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:28.147689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:28.147689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:28.147689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:28.147689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:28.147689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:28.147689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:28.147689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:28.147689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:28.147689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:06:28.147689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:28.149113 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 468/928] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 31 19:06:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:28.154163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a56011-33d8-4b5f-91eb-a42c884ba6fc None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:28.188554 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:28.209206 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffe54a-1e1a-4697-a419-86c1e4e37574 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:28.254726 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0dade-470f-4e8a-9bc8-b407c74850ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:28.255298 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0dade-470f-4e8a-9bc8-b407c74850ca None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:28.255613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0dade-470f-4e8a-9bc8-b407c74850ca None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:28.255856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0dade-470f-4e8a-9bc8-b407c74850ca None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:28.256508 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0dade-470f-4e8a-9bc8-b407c74850ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:28.256816 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0dade-470f-4e8a-9bc8-b407c74850ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:28.258202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0dade-470f-4e8a-9bc8-b407c74850ca None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:28.271655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 461/929] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:28.272315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0dade-470f-4e8a-9bc8-b407c74850ca None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:28.288034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a56011-33d8-4b5f-91eb-a42c884ba6fc None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:28.328247 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751c510-bb19-4763-8482-a0106cf4f22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:28.328859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751c510-bb19-4763-8482-a0106cf4f22c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:28.329137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751c510-bb19-4763-8482-a0106cf4f22c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:28.329405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751c510-bb19-4763-8482-a0106cf4f22c None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:28.330170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751c510-bb19-4763-8482-a0106cf4f22c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:28.330472 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751c510-bb19-4763-8482-a0106cf4f22c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:28.331872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751c510-bb19-4763-8482-a0106cf4f22c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:28.336964 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 469/930] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 31 19:06:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:28.337812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751c510-bb19-4763-8482-a0106cf4f22c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:28.386342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:28.407355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0dade-470f-4e8a-9bc8-b407c74850ca None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:28.444813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9994d2-9313-43e5-919e-05dcd648bf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:28.445447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9994d2-9313-43e5-919e-05dcd648bf00 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:28.445721 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9994d2-9313-43e5-919e-05dcd648bf00 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:28.445989 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9994d2-9313-43e5-919e-05dcd648bf00 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:28.446562 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9994d2-9313-43e5-919e-05dcd648bf00 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:28.448008 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9994d2-9313-43e5-919e-05dcd648bf00 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:28.451632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751c510-bb19-4763-8482-a0106cf4f22c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:28.458137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-9f9994d2-9313-43e5-919e-05dcd648bf00 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:06:28.458137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:28.458137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:28.458137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:28.458137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:28.458137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:28.458137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:28.458137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:28.458137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:06:28.458137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:28.458137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:28.458137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:28.458137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:28.458137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:28.458137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:28.458137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:28.463569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:28.463569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:28.463569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:28.463569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:28.463569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:28.463569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:28.463569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:28.463569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:28.463569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:28.463569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:28.463569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:28.463569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:28.463569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:28.463569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:28.463569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:28.463569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:28.463569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:28.464718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:28.464718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:28.464718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:28.464718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:28.464718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:28.464718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:28.464718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:28.464718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:28.464718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:28.464718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:28.464718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:28.464718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:28.464718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:06:28.464718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:28.464718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 462/931] 10.222.0.59 () {56 vars in 1132 bytes} [Wed Aug 31 19:06:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:28.465616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9994d2-9313-43e5-919e-05dcd648bf00 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:28.478639 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:28.500004 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db7a82-4376-49d1-a785-f0db14d97688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:28.500645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db7a82-4376-49d1-a785-f0db14d97688 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:28.500926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db7a82-4376-49d1-a785-f0db14d97688 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:28.501198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db7a82-4376-49d1-a785-f0db14d97688 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:28.509958 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db7a82-4376-49d1-a785-f0db14d97688 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:28.511379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db7a82-4376-49d1-a785-f0db14d97688 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:28.523517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-99db7a82-4376-49d1-a785-f0db14d97688 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:06:28.523517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:28.523517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:28.523517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:28.523517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:28.523517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:28.523517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:28.523517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:28.523517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:28.523517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:28.523517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:28.523517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:28.523517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:06:28.523517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:06:28.523517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:06:28.523517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:28.524534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:28.525595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:28.525595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:28.525595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:06:28.525595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:06:28.525595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:06:28.525595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:06:28.525595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:06:28.525595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:06:28.525595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:06:28.525595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:28.525595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 470/932] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 31 19:06:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:28.525595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db7a82-4376-49d1-a785-f0db14d97688 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:28.590815 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9994d2-9313-43e5-919e-05dcd648bf00 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:28.632491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7efc7c1-d646-483b-973c-2b0d67c673ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:28.633103 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efc7c1-d646-483b-973c-2b0d67c673ac None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:28.633388 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efc7c1-d646-483b-973c-2b0d67c673ac None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:28.633663 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efc7c1-d646-483b-973c-2b0d67c673ac None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:28.634296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7efc7c1-d646-483b-973c-2b0d67c673ac None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:28.634632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7efc7c1-d646-483b-973c-2b0d67c673ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:28.636014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7efc7c1-d646-483b-973c-2b0d67c673ac None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:28.638478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:28.641901 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db7a82-4376-49d1-a785-f0db14d97688 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:28.642574 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 463/933] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 31 19:06:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:28.643633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7efc7c1-d646-483b-973c-2b0d67c673ac None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:28.682747 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ed515-dc95-4aa7-8074-a222d57e3890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:28.683450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ed515-dc95-4aa7-8074-a222d57e3890 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:28.683675 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ed515-dc95-4aa7-8074-a222d57e3890 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:28.684142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ed515-dc95-4aa7-8074-a222d57e3890 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:28.684796 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ed515-dc95-4aa7-8074-a222d57e3890 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:28.685115 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ed515-dc95-4aa7-8074-a222d57e3890 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:28.686500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ed515-dc95-4aa7-8074-a222d57e3890 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:28.697027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 471/934] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 31 19:06:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:28.697027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ed515-dc95-4aa7-8074-a222d57e3890 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:28.737618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:28.766323 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7efc7c1-d646-483b-973c-2b0d67c673ac None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:28.804465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffa31f-5607-4b3a-8210-76688fbb840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:28.805078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffa31f-5607-4b3a-8210-76688fbb840b None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:28.805357 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffa31f-5607-4b3a-8210-76688fbb840b None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:28.805631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffa31f-5607-4b3a-8210-76688fbb840b None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:28.805812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ed515-dc95-4aa7-8074-a222d57e3890 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:28.806407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffa31f-5607-4b3a-8210-76688fbb840b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:28.806718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffa31f-5607-4b3a-8210-76688fbb840b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:28.808215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffa31f-5607-4b3a-8210-76688fbb840b None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:28.833449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 464/935] 10.222.0.59 () {56 vars in 1149 bytes} [Wed Aug 31 19:06:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:28.834828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffa31f-5607-4b3a-8210-76688fbb840b None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:28.846385 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bf92f-2786-477f-b362-c37467e2faf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:28.847097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bf92f-2786-477f-b362-c37467e2faf7 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:28.847382 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bf92f-2786-477f-b362-c37467e2faf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:28.847654 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bf92f-2786-477f-b362-c37467e2faf7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:28.848368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342bf92f-2786-477f-b362-c37467e2faf7 None admin] RBAC: Adding query filter params (interface=public, service_id=d02a9e70b0584f40acbe83ac90ea9041, region_id=RegionOne) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:28.848680 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342bf92f-2786-477f-b362-c37467e2faf7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:28.850052 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342bf92f-2786-477f-b362-c37467e2faf7 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:28.859759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 472/936] 10.222.0.59 () {56 vars in 1251 bytes} [Wed Aug 31 19:06:28 2022] GET /identity/v3/endpoints?service_id=d02a9e70b0584f40acbe83ac90ea9041&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:28.859759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bf92f-2786-477f-b362-c37467e2faf7 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:28.886885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:28.951818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffa31f-5607-4b3a-8210-76688fbb840b None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:28.966479 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bf92f-2786-477f-b362-c37467e2faf7 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:28.983895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:28.989671 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfe64b-c0f7-4e74-b7a2-223b01f461ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:28.990292 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfe64b-c0f7-4e74-b7a2-223b01f461ee None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:28.990571 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfe64b-c0f7-4e74-b7a2-223b01f461ee None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:28.990846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfe64b-c0f7-4e74-b7a2-223b01f461ee None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:28.991502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfe64b-c0f7-4e74-b7a2-223b01f461ee None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:28.991809 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfe64b-c0f7-4e74-b7a2-223b01f461ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:28.993178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfe64b-c0f7-4e74-b7a2-223b01f461ee None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:28.998087 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 465/937] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 31 19:06:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:29.003677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfe64b-c0f7-4e74-b7a2-223b01f461ee None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:29.008114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de6f44-5eff-422c-88a6-98e6ae4f31fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:29.008757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6f44-5eff-422c-88a6-98e6ae4f31fa None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:29.009038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6f44-5eff-422c-88a6-98e6ae4f31fa None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:29.009312 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6f44-5eff-422c-88a6-98e6ae4f31fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:29.021971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de6f44-5eff-422c-88a6-98e6ae4f31fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:29.023552 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de6f44-5eff-422c-88a6-98e6ae4f31fa None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:29.039353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-21de6f44-5eff-422c-88a6-98e6ae4f31fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:29.039353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:29.039353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:29.039353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:29.039353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:29.039353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:29.039353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:29.039353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:29.039353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:29.039353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:29.039353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:29.039353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:29.039353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:29.039353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:29.039353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:29.039353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:29.040384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:29.040384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:29.040384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:29.040384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:29.040384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:29.040384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:29.040384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:29.040384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:29.040384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:29.040384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:29.040384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:29.040384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:29.040384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:29.040384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:29.040384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:29.040384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:29.040384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:29.041343 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:29.041343 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:29.041343 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:29.045278 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 473/938] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:29.045981 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de6f44-5eff-422c-88a6-98e6ae4f31fa None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:29.110232 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfe64b-c0f7-4e74-b7a2-223b01f461ee None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:29.135479 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:29.150213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f64eb3-053e-4828-8240-cb4cd9f2241f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:29.150829 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f64eb3-053e-4828-8240-cb4cd9f2241f None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:29.151154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f64eb3-053e-4828-8240-cb4cd9f2241f None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:29.151439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f64eb3-053e-4828-8240-cb4cd9f2241f None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:29.152045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f64eb3-053e-4828-8240-cb4cd9f2241f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:29.153418 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f64eb3-053e-4828-8240-cb4cd9f2241f None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:29.156504 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de6f44-5eff-422c-88a6-98e6ae4f31fa None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:29.171269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 466/939] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 31 19:06:28 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:06:29.171269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f64eb3-053e-4828-8240-cb4cd9f2241f None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:29.222293 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd41d2-6858-4ef0-983f-3e9f669876d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:29.223152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd41d2-6858-4ef0-983f-3e9f669876d3 None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:29.223508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd41d2-6858-4ef0-983f-3e9f669876d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:29.223858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd41d2-6858-4ef0-983f-3e9f669876d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:29.224630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd41d2-6858-4ef0-983f-3e9f669876d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:29.227023 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd41d2-6858-4ef0-983f-3e9f669876d3 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:29.253675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:29.308059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f64eb3-053e-4828-8240-cb4cd9f2241f None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:29.351466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 474/940] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:06:29 2022] POST /identity/v3/endpoints => generated 347 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:29.361715 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afa049-ec3e-4af5-9e7d-6821429f3ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:29.362722 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afa049-ec3e-4af5-9e7d-6821429f3ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:29.363201 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afa049-ec3e-4af5-9e7d-6821429f3ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:29.363642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afa049-ec3e-4af5-9e7d-6821429f3ec8 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:29.364472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afa049-ec3e-4af5-9e7d-6821429f3ec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:29.364927 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afa049-ec3e-4af5-9e7d-6821429f3ec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:29.366576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afa049-ec3e-4af5-9e7d-6821429f3ec8 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:29.383490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 467/941] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:29.389756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd41d2-6858-4ef0-983f-3e9f669876d3 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:29.520171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd41d2-6858-4ef0-983f-3e9f669876d3 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:29.560375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e955612-c705-484f-bc11-8a22b8807cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:29.560938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e955612-c705-484f-bc11-8a22b8807cdf None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:29.561224 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e955612-c705-484f-bc11-8a22b8807cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:29.561499 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e955612-c705-484f-bc11-8a22b8807cdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:29.564819 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:29.569502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e955612-c705-484f-bc11-8a22b8807cdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:29.570995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e955612-c705-484f-bc11-8a22b8807cdf None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:29.583917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-4e955612-c705-484f-bc11-8a22b8807cdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:29.583917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:29.583917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:29.583917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:29.583917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:29.583917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:29.583917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:29.583917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:29.583917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:29.583917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:29.583917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:29.583917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:29.583917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:06:29.583917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:06:29.583917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:06:29.585165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:06:29.585165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:29.585165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:29.585165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:29.585165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:29.585165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:29.585165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:29.585165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:29.585165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:29.585165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:29.585165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:29.585165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:29.585165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:29.585165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:29.585165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:29.585165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:29.585165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:29.586498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:06:29.586498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:06:29.586498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:06:29.586498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:06:29.586498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:06:29.586498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:06:29.586498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:29.586498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:29.586498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 475/942] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:29.592940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afa049-ec3e-4af5-9e7d-6821429f3ec8 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:29.713808 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afa049-ec3e-4af5-9e7d-6821429f3ec8 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:29.732424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:29.751980 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e26b71-633c-490d-b821-f36f3e836db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:29.752751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e26b71-633c-490d-b821-f36f3e836db4 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:29.753130 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e26b71-633c-490d-b821-f36f3e836db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:29.753502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e26b71-633c-490d-b821-f36f3e836db4 None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:29.754301 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e26b71-633c-490d-b821-f36f3e836db4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:29.754725 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e26b71-633c-490d-b821-f36f3e836db4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:29.756270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e26b71-633c-490d-b821-f36f3e836db4 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:29.769959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 468/943] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:06:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:29.776102 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e955612-c705-484f-bc11-8a22b8807cdf None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:29.841030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6705e-777c-41c4-be98-6ad92d281b87 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:29.841384 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6705e-777c-41c4-be98-6ad92d281b87 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:29.841634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6705e-777c-41c4-be98-6ad92d281b87 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:29.844225 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 469/944] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:29.849127 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd2262-8495-4340-bdf0-db89981c296d None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:29.849508 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd2262-8495-4340-bdf0-db89981c296d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:29.849734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd2262-8495-4340-bdf0-db89981c296d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:29.870958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-2fbd2262-8495-4340-bdf0-db89981c296d None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:29.872534 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-2fbd2262-8495-4340-bdf0-db89981c296d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:29.899340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e955612-c705-484f-bc11-8a22b8807cdf None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:29.943297 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263c257-861b-4775-b7c4-3e406e9981d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:29.944280 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263c257-861b-4775-b7c4-3e406e9981d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:29.944565 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263c257-861b-4775-b7c4-3e406e9981d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:29.944837 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263c257-861b-4775-b7c4-3e406e9981d9 None admin] PATH_INFO: `/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/1ee990b1658e49cc8bc2fe15e7dc36d9/roles/e22b60b1c78d45b29c81bcce9064de49` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:29.947557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d263c257-861b-4775-b7c4-3e406e9981d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=161c90d34bb34925a1aaeb70c7de7eae, user_id=1ee990b1658e49cc8bc2fe15e7dc36d9, role_id=e22b60b1c78d45b29c81bcce9064de49)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:29.949288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d263c257-861b-4775-b7c4-3e406e9981d9 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:29.962309 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 470/945] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:30.006960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 476/946] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:06:29 2022] PUT /identity/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/1ee990b1658e49cc8bc2fe15e7dc36d9/roles/e22b60b1c78d45b29c81bcce9064de49 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:06:30.248232 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd2262-8495-4340-bdf0-db89981c296d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:30.360458 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd2262-8495-4340-bdf0-db89981c296d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:30.368906 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:30.398834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e40f1-ce3c-4554-b485-8a1a3cf2de8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:30.399495 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e40f1-ce3c-4554-b485-8a1a3cf2de8c None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:30.399780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e40f1-ce3c-4554-b485-8a1a3cf2de8c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:30.400059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e40f1-ce3c-4554-b485-8a1a3cf2de8c None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:30.400857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e40f1-ce3c-4554-b485-8a1a3cf2de8c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:30.402819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e40f1-ce3c-4554-b485-8a1a3cf2de8c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:30.405670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: WARNING keystone.server.flask.common [None req-090e40f1-ce3c-4554-b485-8a1a3cf2de8c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 19:06:30.453531 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.sql.core [None req-090e40f1-ce3c-4554-b485-8a1a3cf2de8c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 19:06:30.453531 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 19:06:30.453531 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [parameters: {'id': 'ef8d740997a74e97a1b85a758ca23f79', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 19:06:30.453531 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81869) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 19:06:30.472735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-090e40f1-ce3c-4554-b485-8a1a3cf2de8c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ef8d740997a74e97a1b85a758ca23f79.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ef8d740997a74e97a1b85a758ca23f79. Aug 31 19:06:30.472735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:30.472735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 19:06:30.472735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 19:06:30.472735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 19:06:30.472735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 19:06:30.472735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 19:06:30.472735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 19:06:30.472735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 19:06:30.472735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application conn.query(q) Aug 31 19:06:30.472735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 19:06:30.472735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 19:06:30.472735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application result.read() Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 31 19:06:30.473676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 31 19:06:30.474696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 19:06:30.474696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application next(self.gen) Aug 31 19:06:30.474696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 19:06:30.474696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application yield resource Aug 31 19:06:30.474696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 19:06:30.474696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application next(self.gen) Aug 31 19:06:30.474696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 19:06:30.474696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 19:06:30.474696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 19:06:30.474696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 19:06:30.474696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 19:06:30.474696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise self.value Aug 31 19:06:30.474696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 19:06:30.474696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 19:06:30.474696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 19:06:30.474696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application session.commit() Aug 31 19:06:30.474696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 19:06:30.475642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 19:06:30.475642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 19:06:30.475642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 19:06:30.475642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 19:06:30.475642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self.session.flush() Aug 31 19:06:30.475642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 19:06:30.475642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 19:06:30.475642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 19:06:30.475642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 19:06:30.475642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 19:06:30.475642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application compat.raise_( Aug 31 19:06:30.475642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 19:06:30.475642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception Aug 31 19:06:30.475642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 19:06:30.475642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 19:06:30.475642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 19:06:30.475642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 19:06:30.476567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 19:06:30.476567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 19:06:30.476567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 19:06:30.476567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 19:06:30.476567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 19:06:30.476567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 19:06:30.476567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 19:06:30.476567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 19:06:30.476567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 19:06:30.476567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 19:06:30.476567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 19:06:30.476567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 19:06:30.476567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 19:06:30.476567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 19:06:30.476567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 19:06:30.477382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 19:06:30.477382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 19:06:30.477382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception Aug 31 19:06:30.477382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 19:06:30.477382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 19:06:30.477382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 19:06:30.477382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 19:06:30.477382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 19:06:30.477382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 19:06:30.477382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 19:06:30.477382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application conn.query(q) Aug 31 19:06:30.477382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 19:06:30.477382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 19:06:30.477382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 19:06:30.477382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application result.read() Aug 31 19:06:30.477382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 19:06:30.477382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 19:06:30.478284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 19:06:30.478284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 19:06:30.478284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 19:06:30.478284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 19:06:30.478284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 19:06:30.478284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 19:06:30.478284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 19:06:30.478284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 19:06:30.478284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [parameters: {'id': 'ef8d740997a74e97a1b85a758ca23f79', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 19:06:30.478284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 19:06:30.478284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:30.478284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:30.478284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:30.478284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:30.478284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 31 19:06:30.480369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 31 19:06:30.480369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 19:06:30.480369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 19:06:30.480369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 31 19:06:30.480369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:30.480369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:30.480369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:30.480369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:30.480369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:30.480369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:30.480369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:30.480369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:30.480369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:30.480369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:30.480369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:30.480369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:30.480369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:30.482621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:30.482621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 31 19:06:30.482621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 31 19:06:30.482621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:30.482621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:30.482621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 31 19:06:30.482621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 31 19:06:30.482621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ef8d740997a74e97a1b85a758ca23f79. Aug 31 19:06:30.482621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:30.482621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 471/947] 10.222.0.59 () {60 vars in 1140 bytes} [Wed Aug 31 19:06:30 2022] POST /identity/v3/projects => generated 262 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:30.486563 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263c257-861b-4775-b7c4-3e406e9981d9 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:30.552245 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:30.606985 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263c257-861b-4775-b7c4-3e406e9981d9 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:30.646342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165a0b3-d48c-4802-9206-70400d4af621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:30.647008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165a0b3-d48c-4802-9206-70400d4af621 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:30.647288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165a0b3-d48c-4802-9206-70400d4af621 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:30.647559 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165a0b3-d48c-4802-9206-70400d4af621 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:30.654721 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2165a0b3-d48c-4802-9206-70400d4af621 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:30.656272 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2165a0b3-d48c-4802-9206-70400d4af621 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:30.668529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-2165a0b3-d48c-4802-9206-70400d4af621 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:30.668529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:30.668529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:30.668529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:30.668529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:30.668529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:30.668529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:30.668529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:30.668529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:30.668529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:30.668529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:30.668529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:30.668529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:06:30.668529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:06:30.668529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:06:30.669505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:06:30.669505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:30.669505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:30.669505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:30.669505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:30.669505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:30.669505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:30.669505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:30.669505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:30.669505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:30.669505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:30.669505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:30.669505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:30.669505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:30.669505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:30.669505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:30.669505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:30.670410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:06:30.670410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:06:30.670410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:06:30.670410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:06:30.670410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:06:30.670410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:06:30.670410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:30.670410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:30.670410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 477/948] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:30.677090 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e40f1-ce3c-4554-b485-8a1a3cf2de8c None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:30.800111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e40f1-ce3c-4554-b485-8a1a3cf2de8c None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:30.836959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2cdfa-10bd-4865-b2b7-a5210a259441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:30.837425 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2cdfa-10bd-4865-b2b7-a5210a259441 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:30.837706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2cdfa-10bd-4865-b2b7-a5210a259441 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:30.837983 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2cdfa-10bd-4865-b2b7-a5210a259441 None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:30.838657 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2cdfa-10bd-4865-b2b7-a5210a259441 None admin] RBAC: Adding query filter params (name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:30.838992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2cdfa-10bd-4865-b2b7-a5210a259441 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:30.840626 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2cdfa-10bd-4865-b2b7-a5210a259441 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:30.864039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 472/949] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 31 19:06:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:31.300800 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a7afa-1725-4cc1-aa03-39702517b86f None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:31.301335 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a7afa-1725-4cc1-aa03-39702517b86f None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:31.301335 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a7afa-1725-4cc1-aa03-39702517b86f None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:31.302970 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 478/950] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:31.308695 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2cb81-8457-4c3c-8058-3ebd318563b0 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:31.309059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2cb81-8457-4c3c-8058-3ebd318563b0 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:31.309435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2cb81-8457-4c3c-8058-3ebd318563b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:31.332438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-b6d2cb81-8457-4c3c-8058-3ebd318563b0 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:31.333979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b6d2cb81-8457-4c3c-8058-3ebd318563b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:31.352358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:31.408015 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 473/951] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:31.454122 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8937c-0e73-4025-98e6-f1541f7630fc None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:31.455093 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8937c-0e73-4025-98e6-f1541f7630fc None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:31.455400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8937c-0e73-4025-98e6-f1541f7630fc None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:31.457011 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 479/952] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:31.462073 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf12d6-1b95-4c9a-bc0a-701207c5bc65 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:31.462434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf12d6-1b95-4c9a-bc0a-701207c5bc65 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:31.462642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf12d6-1b95-4c9a-bc0a-701207c5bc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:31.481913 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-dbaf12d6-1b95-4c9a-bc0a-701207c5bc65 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:31.483132 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-dbaf12d6-1b95-4c9a-bc0a-701207c5bc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:31.557598 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 474/953] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:31.679571 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8937c-0e73-4025-98e6-f1541f7630fc None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:31.790455 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8937c-0e73-4025-98e6-f1541f7630fc None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:31.809525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:31.828903 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22599611-c89a-4c8e-870d-da4cc683f011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:31.829713 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22599611-c89a-4c8e-870d-da4cc683f011 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:31.830205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22599611-c89a-4c8e-870d-da4cc683f011 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:31.830617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22599611-c89a-4c8e-870d-da4cc683f011 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:31.831238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22599611-c89a-4c8e-870d-da4cc683f011 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:31.832618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22599611-c89a-4c8e-870d-da4cc683f011 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:31.841904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-22599611-c89a-4c8e-870d-da4cc683f011 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:06:31.841904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:31.841904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:31.841904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:31.841904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:31.841904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:31.841904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:31.841904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:31.841904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:06:31.841904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:31.841904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:31.841904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:31.841904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:31.841904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:31.841904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:31.841904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:31.841904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:31.844441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:31.844441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:31.844441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:31.844441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:31.844441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:31.844441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:31.844441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:31.844441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:31.844441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:31.844441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:31.844441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:31.844441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:31.844441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:31.844441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:31.844441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:31.844441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:31.846143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:31.846143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:31.846143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:31.846143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:31.846143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:31.846143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:31.846143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:31.846143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:31.846143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:31.846143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:31.846143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:31.846143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:31.846143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:06:31.846143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:31.847985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf12d6-1b95-4c9a-bc0a-701207c5bc65 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:31.848485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 480/954] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 31 19:06:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:31.851845 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22599611-c89a-4c8e-870d-da4cc683f011 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:31.956251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:31.961414 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22599611-c89a-4c8e-870d-da4cc683f011 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:31.982380 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf12d6-1b95-4c9a-bc0a-701207c5bc65 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:32.008854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af5f53-ab07-4587-866f-7047d2e237a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:32.010160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af5f53-ab07-4587-866f-7047d2e237a5 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:32.010850 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af5f53-ab07-4587-866f-7047d2e237a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:32.011293 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af5f53-ab07-4587-866f-7047d2e237a5 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:32.012345 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af5f53-ab07-4587-866f-7047d2e237a5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:32.013218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af5f53-ab07-4587-866f-7047d2e237a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:32.015305 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af5f53-ab07-4587-866f-7047d2e237a5 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:32.021739 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe333a0-e960-4fd7-bc86-4e0940708450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:32.022471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe333a0-e960-4fd7-bc86-4e0940708450 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:32.022900 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe333a0-e960-4fd7-bc86-4e0940708450 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:32.023325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe333a0-e960-4fd7-bc86-4e0940708450 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:32.023670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 481/955] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 31 19:06:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:32.024933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe333a0-e960-4fd7-bc86-4e0940708450 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:32.026472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe333a0-e960-4fd7-bc86-4e0940708450 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:32.029129 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af5f53-ab07-4587-866f-7047d2e237a5 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:32.039992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-ebe333a0-e960-4fd7-bc86-4e0940708450 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:06:32.039992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:32.039992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:32.039992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:32.039992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:32.039992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:32.039992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:32.039992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:32.039992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:06:32.039992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:32.039992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:32.039992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:32.039992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:32.039992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:32.039992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:32.039992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:32.039992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:32.041094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:32.041094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:32.041094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:32.041094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:32.041094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:32.041094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:32.041094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:32.041094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:32.041094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:32.041094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:32.041094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:32.041094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:32.041094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:32.041094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:32.041094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:32.041094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:32.041970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:32.041970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:32.041970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:32.041970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:32.041970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:32.041970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:32.041970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:32.041970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:32.041970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:32.041970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:32.041970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:32.041970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:32.041970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:06:32.041970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:32.050530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 475/956] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:32.051911 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe333a0-e960-4fd7-bc86-4e0940708450 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:32.090139 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:32.158078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af5f53-ab07-4587-866f-7047d2e237a5 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:32.167994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe333a0-e960-4fd7-bc86-4e0940708450 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:32.199336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5cf4c-cdb5-4b60-a208-7748ad633f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:32.199952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5cf4c-cdb5-4b60-a208-7748ad633f33 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:32.200235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5cf4c-cdb5-4b60-a208-7748ad633f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:32.200571 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5cf4c-cdb5-4b60-a208-7748ad633f33 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:32.201231 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5cf4c-cdb5-4b60-a208-7748ad633f33 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:32.201538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5cf4c-cdb5-4b60-a208-7748ad633f33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:32.202933 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5cf4c-cdb5-4b60-a208-7748ad633f33 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:32.213453 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 482/957] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 31 19:06:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:32.213620 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c27f6f-c7cb-4849-8080-0a3331e3a693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:32.213620 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c27f6f-c7cb-4849-8080-0a3331e3a693 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:32.213819 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e2f23-fe49-4f44-aa9c-8bab5810694c None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:32.213893 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c27f6f-c7cb-4849-8080-0a3331e3a693 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:32.213973 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e2f23-fe49-4f44-aa9c-8bab5810694c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:32.214053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c27f6f-c7cb-4849-8080-0a3331e3a693 None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:32.214124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e2f23-fe49-4f44-aa9c-8bab5810694c None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:32.214195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c27f6f-c7cb-4849-8080-0a3331e3a693 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:32.214195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c27f6f-c7cb-4849-8080-0a3331e3a693 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:32.214195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c27f6f-c7cb-4849-8080-0a3331e3a693 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:32.215862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 483/958] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:32.216638 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e2f23-fe49-4f44-aa9c-8bab5810694c None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:32.223761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 476/959] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 31 19:06:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:32.223761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187d57e-05e4-43be-97f7-38e318038d9b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:32.223761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187d57e-05e4-43be-97f7-38e318038d9b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:32.223761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187d57e-05e4-43be-97f7-38e318038d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:32.241436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-3187d57e-05e4-43be-97f7-38e318038d9b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:32.242664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-3187d57e-05e4-43be-97f7-38e318038d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:32.246762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:32.331064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 477/960] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:32.332355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187d57e-05e4-43be-97f7-38e318038d9b None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:32.338630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e2f23-fe49-4f44-aa9c-8bab5810694c None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:32.351334 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:32.378458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384f4b8-8062-4b25-9847-9672e0a69e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:32.379105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384f4b8-8062-4b25-9847-9672e0a69e30 None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:32.379388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384f4b8-8062-4b25-9847-9672e0a69e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:32.379664 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384f4b8-8062-4b25-9847-9672e0a69e30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:32.380342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384f4b8-8062-4b25-9847-9672e0a69e30 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:32.381809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384f4b8-8062-4b25-9847-9672e0a69e30 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:32.445945 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187d57e-05e4-43be-97f7-38e318038d9b None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:32.464814 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 484/961] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:06:32 2022] POST /identity/v3/endpoints => generated 349 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:32.496162 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552143c-e8ed-42ce-9d93-ddd5a71b16c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:32.497055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552143c-e8ed-42ce-9d93-ddd5a71b16c3 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:32.497489 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552143c-e8ed-42ce-9d93-ddd5a71b16c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:32.497916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552143c-e8ed-42ce-9d93-ddd5a71b16c3 None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:32.498851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552143c-e8ed-42ce-9d93-ddd5a71b16c3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:32.499362 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552143c-e8ed-42ce-9d93-ddd5a71b16c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:32.500952 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552143c-e8ed-42ce-9d93-ddd5a71b16c3 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:32.504442 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:32.508841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 478/962] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 31 19:06:32 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:32.607228 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384f4b8-8062-4b25-9847-9672e0a69e30 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:32.733032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384f4b8-8062-4b25-9847-9672e0a69e30 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:32.770341 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e471922-a096-495e-a632-49d68486f732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:32.771081 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e471922-a096-495e-a632-49d68486f732 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:32.771365 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e471922-a096-495e-a632-49d68486f732 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:32.771726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e471922-a096-495e-a632-49d68486f732 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:32.784218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e471922-a096-495e-a632-49d68486f732 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:32.789651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-7e471922-a096-495e-a632-49d68486f732 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:06:32.789651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:32.789651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:32.789651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:32.789651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:32.789651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:32.789651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:32.789651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:32.789651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:32.789651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:32.789651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:32.789651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:32.789651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:32.789651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:32.789651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:32.789651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:32.790616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:32.790616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:32.790616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:32.790616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:32.790616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:32.790616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:32.790616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:32.790616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:32.790616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:32.790616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:32.790616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:32.790616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:32.790616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:32.790616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:32.790616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:32.790616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:32.791528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:32.791528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:32.791528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:32.791528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:32.791528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:32.791528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:32.791528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:32.791528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:32.791528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:32.791528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:32.791528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:06:32.791528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:32.791528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 485/963] 10.222.0.59 () {56 vars in 1120 bytes} [Wed Aug 31 19:06:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:32.800357 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552143c-e8ed-42ce-9d93-ddd5a71b16c3 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:32.920810 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552143c-e8ed-42ce-9d93-ddd5a71b16c3 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:32.927565 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e048943-929d-40b7-81fe-6821a7aad7e0 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:32.927852 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e048943-929d-40b7-81fe-6821a7aad7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:32.928133 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e048943-929d-40b7-81fe-6821a7aad7e0 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:32.930693 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 486/964] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:32.934908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7b367-022a-4889-926e-420ca0bf5c50 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:32.935465 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7b367-022a-4889-926e-420ca0bf5c50 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:32.935748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7b367-022a-4889-926e-420ca0bf5c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:32.955190 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-02f7b367-022a-4889-926e-420ca0bf5c50 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:32.956397 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-02f7b367-022a-4889-926e-420ca0bf5c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:32.958369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff0178-ee56-43a1-8f25-c38add1296da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:32.959028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff0178-ee56-43a1-8f25-c38add1296da None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:32.959313 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff0178-ee56-43a1-8f25-c38add1296da None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:32.959586 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff0178-ee56-43a1-8f25-c38add1296da None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:32.960265 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff0178-ee56-43a1-8f25-c38add1296da None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:32.960569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff0178-ee56-43a1-8f25-c38add1296da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:32.961933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff0178-ee56-43a1-8f25-c38add1296da None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:32.971943 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 479/965] 10.222.0.59 () {56 vars in 1115 bytes} [Wed Aug 31 19:06:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:32.978842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff0178-ee56-43a1-8f25-c38add1296da None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:32.980826 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:33.053751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 487/966] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:33.063439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:33.106723 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff0178-ee56-43a1-8f25-c38add1296da None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:33.144202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241972fe-14cb-4e4d-9c58-715f52a2d242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:33.145211 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241972fe-14cb-4e4d-9c58-715f52a2d242 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:33.145691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241972fe-14cb-4e4d-9c58-715f52a2d242 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:33.146161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241972fe-14cb-4e4d-9c58-715f52a2d242 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:33.157874 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241972fe-14cb-4e4d-9c58-715f52a2d242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:33.159717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241972fe-14cb-4e4d-9c58-715f52a2d242 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:33.175869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-241972fe-14cb-4e4d-9c58-715f52a2d242 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:33.175869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:33.175869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:33.175869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:33.175869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:33.175869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:33.175869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:33.175869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:33.175869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:33.175869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:33.175869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:33.175869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:33.175869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:33.175869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:33.175869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:33.175869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:33.176898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:33.176898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:33.176898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:33.176898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:33.176898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:33.176898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:33.176898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:33.176898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:33.176898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:33.176898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:33.176898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:33.176898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:33.176898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:33.176898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:33.176898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:33.176898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:33.176898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:33.177837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:33.177837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:33.177837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:33.179740 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 480/967] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:33.185854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7b367-022a-4889-926e-420ca0bf5c50 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:33.295592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7b367-022a-4889-926e-420ca0bf5c50 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:33.333378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b9096-8424-4659-a7cc-2e47f73c635a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:33.333996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b9096-8424-4659-a7cc-2e47f73c635a None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:33.334281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b9096-8424-4659-a7cc-2e47f73c635a None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:33.334557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b9096-8424-4659-a7cc-2e47f73c635a None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:33.335243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b9096-8424-4659-a7cc-2e47f73c635a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:33.335550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b9096-8424-4659-a7cc-2e47f73c635a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:33.336851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241972fe-14cb-4e4d-9c58-715f52a2d242 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:33.337405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b9096-8424-4659-a7cc-2e47f73c635a None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:33.352135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 488/968] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:33.357660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b9096-8424-4659-a7cc-2e47f73c635a None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:33.375488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:33.470710 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241972fe-14cb-4e4d-9c58-715f52a2d242 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:33.482049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b9096-8424-4659-a7cc-2e47f73c635a None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:33.487504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:33.507454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e3fab-d980-4207-b313-c0158e370488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:33.508223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e3fab-d980-4207-b313-c0158e370488 None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:33.508607 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e3fab-d980-4207-b313-c0158e370488 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:33.508960 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e3fab-d980-4207-b313-c0158e370488 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:33.509690 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e3fab-d980-4207-b313-c0158e370488 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:33.511596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e3fab-d980-4207-b313-c0158e370488 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:33.529690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0ef89-a446-4785-8535-9ca37de07540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:33.530346 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0ef89-a446-4785-8535-9ca37de07540 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:33.530628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0ef89-a446-4785-8535-9ca37de07540 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:33.530929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0ef89-a446-4785-8535-9ca37de07540 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:33.539666 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0ef89-a446-4785-8535-9ca37de07540 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:33.541399 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0ef89-a446-4785-8535-9ca37de07540 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:33.550447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.sql.core [None req-7f6e3fab-d980-4207-b313-c0158e370488 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 19:06:33.550447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 19:06:33.550447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [parameters: {'user_id': '158070f3827746a38f13a6028c6d6d96', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 19:06:33.550447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81869) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 19:06:33.553606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-faf0ef89-a446-4785-8535-9ca37de07540 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:06:33.553606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:33.553606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:33.553606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:33.553606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:33.553606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:33.553606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:33.553606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:33.553606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:33.553606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:33.553606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:33.553606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:33.553606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:06:33.553606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:06:33.553606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:06:33.553606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:33.554635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:33.555668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:33.555668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:33.555668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:06:33.555668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:06:33.555668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:06:33.555668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:06:33.555668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:06:33.555668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:06:33.555668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:06:33.555668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:33.559616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 489/969] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 31 19:06:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:33.560898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0ef89-a446-4785-8535-9ca37de07540 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:33.562494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-7f6e3fab-d980-4207-b313-c0158e370488 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 19:06:33.562494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:33.562494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 19:06:33.562494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 19:06:33.562494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 19:06:33.562494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 19:06:33.562494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 19:06:33.562494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 19:06:33.562494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 19:06:33.562494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application conn.query(q) Aug 31 19:06:33.562494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 19:06:33.562494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 19:06:33.562494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 19:06:33.562494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application result.read() Aug 31 19:06:33.562494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 19:06:33.563427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application next(self.gen) Aug 31 19:06:33.565200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 19:06:33.565200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application yield resource Aug 31 19:06:33.565200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 19:06:33.565200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application next(self.gen) Aug 31 19:06:33.565200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 19:06:33.565200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 19:06:33.565200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 19:06:33.565200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 19:06:33.565200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 19:06:33.565200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise self.value Aug 31 19:06:33.565200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 19:06:33.565200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 19:06:33.565200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 19:06:33.565200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application session.commit() Aug 31 19:06:33.565200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 19:06:33.565200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 19:06:33.565200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 19:06:33.566150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 19:06:33.566150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 19:06:33.566150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self.session.flush() Aug 31 19:06:33.566150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 19:06:33.566150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 19:06:33.566150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 19:06:33.566150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 19:06:33.566150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 19:06:33.566150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application compat.raise_( Aug 31 19:06:33.566150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 19:06:33.566150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception Aug 31 19:06:33.566150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 19:06:33.566150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 19:06:33.566150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 19:06:33.566150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 19:06:33.566150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 19:06:33.566150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 19:06:33.568122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 19:06:33.568122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 19:06:33.568122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 31 19:06:33.568122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 31 19:06:33.568122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 19:06:33.568122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 19:06:33.568122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 19:06:33.568122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 19:06:33.568122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 19:06:33.568122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 19:06:33.568122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 19:06:33.568122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 19:06:33.568122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 19:06:33.568122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 19:06:33.568122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 19:06:33.568122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception Aug 31 19:06:33.576355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 19:06:33.576355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 19:06:33.576355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 19:06:33.576355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 19:06:33.576355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 19:06:33.576355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 19:06:33.576355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 19:06:33.576355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application conn.query(q) Aug 31 19:06:33.576355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 19:06:33.576355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 19:06:33.576355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 19:06:33.576355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application result.read() Aug 31 19:06:33.576355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 19:06:33.576355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 19:06:33.576355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 19:06:33.576355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 19:06:33.578403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 19:06:33.578403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 19:06:33.578403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 19:06:33.578403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 19:06:33.578403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 19:06:33.578403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 19:06:33.578403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [parameters: {'user_id': '158070f3827746a38f13a6028c6d6d96', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 19:06:33.578403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 19:06:33.578403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:33.578403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:33.578403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:33.578403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:33.578403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:33.578403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:33.578403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:33.580702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:33.580702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:33.580702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:33.580702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:33.580702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:33.580702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:33.580702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:33.580702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 31 19:06:33.580702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 31 19:06:33.580702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:33.580702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:33.580702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:06:33.580702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:33.580702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:06:33.580702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:33.580702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 31 19:06:33.580702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 31 19:06:33.583017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 31 19:06:33.583017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 31 19:06:33.583017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 19:06:33.583017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 19:06:33.583017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 19:06:33.583017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:33.583017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 481/970] 10.222.0.59 () {60 vars in 1131 bytes} [Wed Aug 31 19:06:33 2022] POST /identity/v3/users => generated 160 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:33.583017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e3fab-d980-4207-b313-c0158e370488 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:33.661719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:33.695261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0ef89-a446-4785-8535-9ca37de07540 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:33.732715 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e56305-008d-46d4-ac3d-ab58cc722f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:33.733401 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e56305-008d-46d4-ac3d-ab58cc722f38 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:33.733588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e56305-008d-46d4-ac3d-ab58cc722f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:33.733816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e56305-008d-46d4-ac3d-ab58cc722f38 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:33.734659 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e56305-008d-46d4-ac3d-ab58cc722f38 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:33.735048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e56305-008d-46d4-ac3d-ab58cc722f38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:33.737298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e56305-008d-46d4-ac3d-ab58cc722f38 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:33.738233 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e3fab-d980-4207-b313-c0158e370488 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:33.763075 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 490/971] 10.222.0.59 () {56 vars in 1149 bytes} [Wed Aug 31 19:06:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:33.764514 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e56305-008d-46d4-ac3d-ab58cc722f38 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:33.784974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7aab0-c0d9-443a-97d7-6d73ac461bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:33.786243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7aab0-c0d9-443a-97d7-6d73ac461bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:33.787145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7aab0-c0d9-443a-97d7-6d73ac461bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:33.787632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7aab0-c0d9-443a-97d7-6d73ac461bd1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:33.799178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7aab0-c0d9-443a-97d7-6d73ac461bd1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:33.801218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7aab0-c0d9-443a-97d7-6d73ac461bd1 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:33.818195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-7ee7aab0-c0d9-443a-97d7-6d73ac461bd1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:06:33.818195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:33.818195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:33.818195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:33.818195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:33.818195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:33.818195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:33.818195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:33.818195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:33.818195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:33.818195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:33.818195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:33.818195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:06:33.818195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:06:33.818195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:06:33.818195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:33.819259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:33.820240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:33.820240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:33.820240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:06:33.820240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:06:33.820240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:06:33.820240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:06:33.820240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:06:33.820240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:06:33.820240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:06:33.820240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:33.820811 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 482/972] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:06:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:33.826491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7aab0-c0d9-443a-97d7-6d73ac461bd1 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:33.867160 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:33.904668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e56305-008d-46d4-ac3d-ab58cc722f38 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:33.942049 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7aab0-c0d9-443a-97d7-6d73ac461bd1 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:33.947887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:33.956073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b9d0b-b770-4624-98a8-98a44cc9f86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:33.957259 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b9d0b-b770-4624-98a8-98a44cc9f86f None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:33.958238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b9d0b-b770-4624-98a8-98a44cc9f86f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:33.958704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b9d0b-b770-4624-98a8-98a44cc9f86f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:33.985061 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d53c95-9935-4c85-8ee6-03cbc376b102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:33.986438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d53c95-9935-4c85-8ee6-03cbc376b102 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:33.986894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d53c95-9935-4c85-8ee6-03cbc376b102 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:33.987425 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d53c95-9935-4c85-8ee6-03cbc376b102 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:33.988651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d53c95-9935-4c85-8ee6-03cbc376b102 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:33.989353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d53c95-9935-4c85-8ee6-03cbc376b102 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:33.991502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d53c95-9935-4c85-8ee6-03cbc376b102 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:34.035858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b9d0b-b770-4624-98a8-98a44cc9f86f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:34.037501 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b9d0b-b770-4624-98a8-98a44cc9f86f None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:34.047611 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 483/973] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 31 19:06:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:34.165685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 491/974] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:06:33 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:06:34.167259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d53c95-9935-4c85-8ee6-03cbc376b102 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:34.278113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:34.285393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d53c95-9935-4c85-8ee6-03cbc376b102 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:34.325194 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfac38e-81dc-4b93-9c21-981da9fe0d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:34.326218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfac38e-81dc-4b93-9c21-981da9fe0d89 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:34.326738 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfac38e-81dc-4b93-9c21-981da9fe0d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:34.327289 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfac38e-81dc-4b93-9c21-981da9fe0d89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:34.339087 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfac38e-81dc-4b93-9c21-981da9fe0d89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:34.340871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfac38e-81dc-4b93-9c21-981da9fe0d89 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:34.358441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-2bfac38e-81dc-4b93-9c21-981da9fe0d89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:34.358441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:34.358441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:34.358441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:34.358441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:34.358441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:34.358441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:34.358441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:34.358441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:34.358441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:34.358441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:34.358441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:34.358441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:34.358441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:34.358441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:34.358441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:34.359507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:34.359507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:34.359507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:34.359507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:34.359507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:34.359507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:34.359507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:34.359507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:34.359507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:34.359507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:34.359507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:34.359507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:34.359507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:34.359507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:34.359507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:34.359507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:34.359507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:34.360427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:34.360427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:34.360427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:34.363883 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 484/975] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:34.369507 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b9d0b-b770-4624-98a8-98a44cc9f86f None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:34.405073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:34.479061 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b9d0b-b770-4624-98a8-98a44cc9f86f None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:34.515292 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d6db3-39cc-4133-9d82-31455bf17bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:34.515906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d6db3-39cc-4133-9d82-31455bf17bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:34.516186 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d6db3-39cc-4133-9d82-31455bf17bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:34.516457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d6db3-39cc-4133-9d82-31455bf17bc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:34.518381 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d6db3-39cc-4133-9d82-31455bf17bc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:34.518729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d6db3-39cc-4133-9d82-31455bf17bc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:34.520132 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d6db3-39cc-4133-9d82-31455bf17bc5 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:34.529207 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab248-17ba-4dd3-ad3a-f94a7f2abda0 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:34.529455 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab248-17ba-4dd3-ad3a-f94a7f2abda0 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:34.529727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab248-17ba-4dd3-ad3a-f94a7f2abda0 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:34.531093 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 485/976] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:34.538236 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 492/977] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:34.539800 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d6db3-39cc-4133-9d82-31455bf17bc5 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:34.540641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad649769-976b-45c0-846f-de162cd7b4e1 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:34.540939 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad649769-976b-45c0-846f-de162cd7b4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:34.541211 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad649769-976b-45c0-846f-de162cd7b4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:34.559736 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ad649769-976b-45c0-846f-de162cd7b4e1 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:34.561239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ad649769-976b-45c0-846f-de162cd7b4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:34.635099 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 486/978] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:34.636355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a83cc-52ce-4d4f-b45e-e2d87b919832 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:34.636642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a83cc-52ce-4d4f-b45e-e2d87b919832 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:34.636915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a83cc-52ce-4d4f-b45e-e2d87b919832 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:34.638146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 487/979] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:34.644105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:34.644833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf76e69-db8c-431e-8f1c-59a06bf8aa29 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:34.645112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf76e69-db8c-431e-8f1c-59a06bf8aa29 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:34.645383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf76e69-db8c-431e-8f1c-59a06bf8aa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:34.655558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d6db3-39cc-4133-9d82-31455bf17bc5 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:34.667445 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-4cf76e69-db8c-431e-8f1c-59a06bf8aa29 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:34.668685 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4cf76e69-db8c-431e-8f1c-59a06bf8aa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:34.695156 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7fe50-4d03-4e98-bece-8f1d205b4569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:34.695787 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7fe50-4d03-4e98-bece-8f1d205b4569 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:34.696062 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7fe50-4d03-4e98-bece-8f1d205b4569 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:34.696357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7fe50-4d03-4e98-bece-8f1d205b4569 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:34.703099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7fe50-4d03-4e98-bece-8f1d205b4569 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:34.705264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7fe50-4d03-4e98-bece-8f1d205b4569 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:34.719336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-cab7fe50-4d03-4e98-bece-8f1d205b4569 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:34.719336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:34.719336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:34.719336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:34.719336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:34.719336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:34.719336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:34.719336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:34.719336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:34.719336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:34.719336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:34.719336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:34.719336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:06:34.719336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:06:34.719336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:06:34.720338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:06:34.720338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:34.720338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:34.720338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:34.720338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:34.720338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:34.720338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:34.720338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:34.720338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:34.720338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:34.720338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:34.720338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:34.720338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:34.720338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:34.720338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:34.720338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:34.720338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:34.721276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:06:34.721276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:06:34.721276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:06:34.721276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:06:34.721276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:06:34.721276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:06:34.721276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:34.721276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:34.721276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 493/980] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:34.727068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7fe50-4d03-4e98-bece-8f1d205b4569 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:34.745944 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 488/981] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:34.834628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7fe50-4d03-4e98-bece-8f1d205b4569 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:34.872228 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588bd3d-0fdd-44d4-b7e0-733933eb7f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:34.872836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588bd3d-0fdd-44d4-b7e0-733933eb7f01 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:34.873121 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588bd3d-0fdd-44d4-b7e0-733933eb7f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:34.873397 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588bd3d-0fdd-44d4-b7e0-733933eb7f01 None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:34.874099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588bd3d-0fdd-44d4-b7e0-733933eb7f01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:34.874411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588bd3d-0fdd-44d4-b7e0-733933eb7f01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:34.875881 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588bd3d-0fdd-44d4-b7e0-733933eb7f01 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:34.890596 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 494/982] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:06:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:34.901080 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf76e69-db8c-431e-8f1c-59a06bf8aa29 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:34.911648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:34.913653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588bd3d-0fdd-44d4-b7e0-733933eb7f01 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:34.930496 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:35.018009 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf76e69-db8c-431e-8f1c-59a06bf8aa29 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:35.031828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588bd3d-0fdd-44d4-b7e0-733933eb7f01 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:35.054475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd395a81-1a5f-4f0f-820a-5a9acccc9fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:35.055108 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd395a81-1a5f-4f0f-820a-5a9acccc9fee None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:35.055386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd395a81-1a5f-4f0f-820a-5a9acccc9fee None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:35.055657 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd395a81-1a5f-4f0f-820a-5a9acccc9fee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:35.056370 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd395a81-1a5f-4f0f-820a-5a9acccc9fee None admin] RBAC: Adding query filter params (role.id=e22b60b1c78d45b29c81bcce9064de49, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=1ee990b1658e49cc8bc2fe15e7dc36d9) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:35.056675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd395a81-1a5f-4f0f-820a-5a9acccc9fee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:35.058088 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd395a81-1a5f-4f0f-820a-5a9acccc9fee None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:35.064469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 489/983] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:06:34 2022] GET /identity/v3/role_assignments?user.id=1ee990b1658e49cc8bc2fe15e7dc36d9&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=e22b60b1c78d45b29c81bcce9064de49 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:35.064469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd395a81-1a5f-4f0f-820a-5a9acccc9fee None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:35.079267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb453ec-62b7-455e-b96d-4c2336e0af75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:35.079267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb453ec-62b7-455e-b96d-4c2336e0af75 None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:35.079267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb453ec-62b7-455e-b96d-4c2336e0af75 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:35.079267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb453ec-62b7-455e-b96d-4c2336e0af75 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:35.079722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb453ec-62b7-455e-b96d-4c2336e0af75 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:35.079722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb453ec-62b7-455e-b96d-4c2336e0af75 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:35.144159 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:35.151574 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 495/984] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 31 19:06:34 2022] POST /identity/v3/services => generated 242 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:06:35.181274 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd395a81-1a5f-4f0f-820a-5a9acccc9fee None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:35.234002 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740cba1-2777-4e6c-99de-a67460fd4099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:35.234823 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740cba1-2777-4e6c-99de-a67460fd4099 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:35.235427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740cba1-2777-4e6c-99de-a67460fd4099 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:35.236186 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740cba1-2777-4e6c-99de-a67460fd4099 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:35.236786 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740cba1-2777-4e6c-99de-a67460fd4099 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:35.238679 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740cba1-2777-4e6c-99de-a67460fd4099 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:35.248949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-5740cba1-2777-4e6c-99de-a67460fd4099 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:06:35.248949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:35.248949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:35.248949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:35.248949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:35.248949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:35.248949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:35.248949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:35.248949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:06:35.248949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:35.248949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:35.248949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:35.248949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:35.248949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:35.248949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:35.248949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:35.248949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:35.250244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:35.250244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:35.250244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:35.250244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:35.250244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:35.250244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:35.250244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:35.250244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:35.250244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:35.250244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:35.250244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:35.250244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:35.250244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:35.250244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:35.250244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:35.250244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:35.251822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:35.251822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:35.251822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:35.251822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:35.251822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:35.251822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:35.251822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:35.251822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:35.251822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:35.251822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:35.251822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:35.251822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:35.251822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:06:35.251822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:35.251822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 490/985] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 31 19:06:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:35.259315 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb453ec-62b7-455e-b96d-4c2336e0af75 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:35.341252 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:35.373146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb453ec-62b7-455e-b96d-4c2336e0af75 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:35.410954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32515f67-77e7-4162-bac5-0bbcb2aa519b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:35.411506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32515f67-77e7-4162-bac5-0bbcb2aa519b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:35.411789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32515f67-77e7-4162-bac5-0bbcb2aa519b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:35.412063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32515f67-77e7-4162-bac5-0bbcb2aa519b None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:35.412711 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32515f67-77e7-4162-bac5-0bbcb2aa519b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:35.413018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32515f67-77e7-4162-bac5-0bbcb2aa519b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:35.414494 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32515f67-77e7-4162-bac5-0bbcb2aa519b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:35.424921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 496/986] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 31 19:06:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:35.426136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740cba1-2777-4e6c-99de-a67460fd4099 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:35.535738 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740cba1-2777-4e6c-99de-a67460fd4099 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:35.565335 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:35.575299 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149fd6e-3b89-4c87-9c30-2bb7d3f19236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:35.575922 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149fd6e-3b89-4c87-9c30-2bb7d3f19236 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:35.576598 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149fd6e-3b89-4c87-9c30-2bb7d3f19236 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:35.576598 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149fd6e-3b89-4c87-9c30-2bb7d3f19236 None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:35.577095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149fd6e-3b89-4c87-9c30-2bb7d3f19236 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:35.577398 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149fd6e-3b89-4c87-9c30-2bb7d3f19236 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:35.578763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149fd6e-3b89-4c87-9c30-2bb7d3f19236 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:35.585099 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 491/987] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 31 19:06:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:35.590844 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32515f67-77e7-4162-bac5-0bbcb2aa519b None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:35.697217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32515f67-77e7-4162-bac5-0bbcb2aa519b None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:35.732343 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e427be-6c96-4188-8bfb-f64c3c29ed24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:35.732968 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e427be-6c96-4188-8bfb-f64c3c29ed24 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:35.733247 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e427be-6c96-4188-8bfb-f64c3c29ed24 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:35.733523 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e427be-6c96-4188-8bfb-f64c3c29ed24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:35.734247 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e427be-6c96-4188-8bfb-f64c3c29ed24 None admin] RBAC: Adding query filter params (interface=admin, service_id=d02a9e70b0584f40acbe83ac90ea9041, region_id=RegionOne) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:35.734562 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e427be-6c96-4188-8bfb-f64c3c29ed24 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:35.736439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e427be-6c96-4188-8bfb-f64c3c29ed24 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:35.738974 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:35.747281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 497/988] 10.222.0.59 () {56 vars in 1249 bytes} [Wed Aug 31 19:06:35 2022] GET /identity/v3/endpoints?service_id=d02a9e70b0584f40acbe83ac90ea9041&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:35.748232 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149fd6e-3b89-4c87-9c30-2bb7d3f19236 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:35.857533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149fd6e-3b89-4c87-9c30-2bb7d3f19236 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:35.893887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a129e98-6a35-43b5-80b2-95153cb3793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:35.894492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a129e98-6a35-43b5-80b2-95153cb3793a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:35.894775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a129e98-6a35-43b5-80b2-95153cb3793a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:35.895080 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a129e98-6a35-43b5-80b2-95153cb3793a None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:35.895681 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a129e98-6a35-43b5-80b2-95153cb3793a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:35.897052 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a129e98-6a35-43b5-80b2-95153cb3793a None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:35.903650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 492/989] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 31 19:06:35 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:06:37.049713 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd1818-81fd-4498-91b0-aec604aae2a1 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:37.050253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd1818-81fd-4498-91b0-aec604aae2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:37.050253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd1818-81fd-4498-91b0-aec604aae2a1 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:37.051675 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 498/990] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:37.057028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c6aa5-dd7d-478d-b485-aa9839c0234a None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:37.057419 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c6aa5-dd7d-478d-b485-aa9839c0234a None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:37.057592 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c6aa5-dd7d-478d-b485-aa9839c0234a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:37.077117 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-846c6aa5-dd7d-478d-b485-aa9839c0234a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:37.078343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-846c6aa5-dd7d-478d-b485-aa9839c0234a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:37.106566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a532-1ab2-4e79-af96-1b440150d5cc None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:37.106819 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a532-1ab2-4e79-af96-1b440150d5cc None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:37.107092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a532-1ab2-4e79-af96-1b440150d5cc None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:37.108536 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 499/991] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:37.114124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b61beb-391c-40b1-8fa9-f8c5efea8f7a None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:37.114407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b61beb-391c-40b1-8fa9-f8c5efea8f7a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:37.114675 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b61beb-391c-40b1-8fa9-f8c5efea8f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:37.133660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d7b61beb-391c-40b1-8fa9-f8c5efea8f7a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:37.134864 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d7b61beb-391c-40b1-8fa9-f8c5efea8f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:37.137554 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:37.151877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 493/992] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:37.210907 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 500/993] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:37.421786 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c6aa5-dd7d-478d-b485-aa9839c0234a None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:37.483923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b61beb-391c-40b1-8fa9-f8c5efea8f7a None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:37.534106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c6aa5-dd7d-478d-b485-aa9839c0234a None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:37.571457 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98619b6b-daf0-49e7-8740-e81cccd0c327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:37.572105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98619b6b-daf0-49e7-8740-e81cccd0c327 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:37.572389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98619b6b-daf0-49e7-8740-e81cccd0c327 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:37.572661 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98619b6b-daf0-49e7-8740-e81cccd0c327 None admin] PATH_INFO: `/v3/services/image` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:37.573246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98619b6b-daf0-49e7-8740-e81cccd0c327 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:37.574631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98619b6b-daf0-49e7-8740-e81cccd0c327 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:37.584655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-98619b6b-daf0-49e7-8740-e81cccd0c327 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:06:37.584655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:37.584655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:37.584655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:37.584655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:37.584655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:37.584655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:37.584655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:37.584655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:06:37.584655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:37.584655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:37.584655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:37.584655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:37.584655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:37.584655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:37.584655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:37.584655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:37.585767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:37.585767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:37.585767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:37.585767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:37.585767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:37.585767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:37.585767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:37.585767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:37.585767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:37.585767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:37.585767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:37.585767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:37.585767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:37.585767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:37.585767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:37.585767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:37.587468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:37.587468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:37.587468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:37.587468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:37.587468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:37.587468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:37.587468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:37.587468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:37.587468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:37.587468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:37.587468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:37.587468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:37.587468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:06:37.587468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:37.587468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 494/994] 10.222.0.59 () {56 vars in 1105 bytes} [Wed Aug 31 19:06:37 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:37.591770 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98619b6b-daf0-49e7-8740-e81cccd0c327 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:37.594612 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b61beb-391c-40b1-8fa9-f8c5efea8f7a None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:37.638300 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:37.644999 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cd0c8-e8d0-4934-b854-f1fc5acc03c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:37.655764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cd0c8-e8d0-4934-b854-f1fc5acc03c5 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:37.655764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cd0c8-e8d0-4934-b854-f1fc5acc03c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:37.655764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cd0c8-e8d0-4934-b854-f1fc5acc03c5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:37.655764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80cd0c8-e8d0-4934-b854-f1fc5acc03c5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:37.655764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80cd0c8-e8d0-4934-b854-f1fc5acc03c5 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:37.656266 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:37.658377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-b80cd0c8-e8d0-4934-b854-f1fc5acc03c5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:06:37.658377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:37.658377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:37.658377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:37.658377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:37.658377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:37.658377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:37.658377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:37.658377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:06:37.658377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:37.658377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:37.658377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:37.658377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:37.658377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:37.658377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:37.658377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:37.658377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:37.659578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:37.659578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:37.659578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:37.659578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:37.659578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:37.659578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:37.659578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:37.659578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:37.659578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:37.659578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:37.659578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:37.659578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:37.659578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:37.659578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:37.659578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:37.659578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:37.660443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:37.660443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:37.660443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:37.660443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:37.660443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:37.660443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:37.660443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:37.660443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:37.660443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:37.660443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:37.660443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:37.660443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:37.660443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:06:37.660443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:37.660443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 501/995] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:37.666271 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cd0c8-e8d0-4934-b854-f1fc5acc03c5 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:37.707298 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98619b6b-daf0-49e7-8740-e81cccd0c327 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:37.745057 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5abb3-2dba-4252-959e-05b60799ca51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:37.745818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5abb3-2dba-4252-959e-05b60799ca51 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:37.746213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5abb3-2dba-4252-959e-05b60799ca51 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:37.746582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5abb3-2dba-4252-959e-05b60799ca51 None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:37.747352 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5abb3-2dba-4252-959e-05b60799ca51 None admin] RBAC: Adding query filter params (name=image) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:37.747750 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5abb3-2dba-4252-959e-05b60799ca51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:37.749199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5abb3-2dba-4252-959e-05b60799ca51 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:37.754839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 495/996] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:37.760422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5abb3-2dba-4252-959e-05b60799ca51 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:37.781729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cd0c8-e8d0-4934-b854-f1fc5acc03c5 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:37.823956 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8c0c4-9b9d-4e5f-9b2f-9339eeafafa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:37.825047 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8c0c4-9b9d-4e5f-9b2f-9339eeafafa5 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:37.825538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8c0c4-9b9d-4e5f-9b2f-9339eeafafa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:37.825904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8c0c4-9b9d-4e5f-9b2f-9339eeafafa5 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:37.826642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8c0c4-9b9d-4e5f-9b2f-9339eeafafa5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:37.827064 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8c0c4-9b9d-4e5f-9b2f-9339eeafafa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:37.828738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8c0c4-9b9d-4e5f-9b2f-9339eeafafa5 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:37.834962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 502/997] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 31 19:06:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:37.836131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12725c1-4fc7-4e07-918e-0e226b8e52df None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:37.836508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12725c1-4fc7-4e07-918e-0e226b8e52df None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:37.836877 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12725c1-4fc7-4e07-918e-0e226b8e52df None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:37.838184 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 503/998] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:37.840259 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12725c1-4fc7-4e07-918e-0e226b8e52df None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:37.869491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5abb3-2dba-4252-959e-05b60799ca51 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:37.886912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:37.901102 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:37.905772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac79f3-92a9-436b-b64f-975cc8cefc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:37.906406 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac79f3-92a9-436b-b64f-975cc8cefc4e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:37.906685 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac79f3-92a9-436b-b64f-975cc8cefc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:37.906986 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac79f3-92a9-436b-b64f-975cc8cefc4e None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:37.908395 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac79f3-92a9-436b-b64f-975cc8cefc4e None admin] RBAC: Adding query filter params (type=image) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:37.908697 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac79f3-92a9-436b-b64f-975cc8cefc4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:37.910061 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac79f3-92a9-436b-b64f-975cc8cefc4e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:37.914515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 496/999] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:37 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:37.915720 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555422db-4d04-41b7-85f4-7b13290e3d43 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:37.917395 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555422db-4d04-41b7-85f4-7b13290e3d43 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:37.917746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555422db-4d04-41b7-85f4-7b13290e3d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:37.938010 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-555422db-4d04-41b7-85f4-7b13290e3d43 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:37.939436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-555422db-4d04-41b7-85f4-7b13290e3d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:37.949375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12725c1-4fc7-4e07-918e-0e226b8e52df None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:37.990944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9f115-119d-45e4-9080-83ff7fc3894b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:37.991593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9f115-119d-45e4-9080-83ff7fc3894b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:37.991959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9f115-119d-45e4-9080-83ff7fc3894b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:37.992339 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9f115-119d-45e4-9080-83ff7fc3894b None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:37.993006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b9f115-119d-45e4-9080-83ff7fc3894b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:37.993313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b9f115-119d-45e4-9080-83ff7fc3894b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:37.994744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b9f115-119d-45e4-9080-83ff7fc3894b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:38.000798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 504/1000] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 31 19:06:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:38.006682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9f115-119d-45e4-9080-83ff7fc3894b None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:38.025177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 497/1001] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:38.123951 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9f115-119d-45e4-9080-83ff7fc3894b None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:38.150266 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:38.161709 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de01bc-f77c-4c67-80cf-4ee52bea3ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:38.162328 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de01bc-f77c-4c67-80cf-4ee52bea3ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:38.162610 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de01bc-f77c-4c67-80cf-4ee52bea3ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:38.164980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de01bc-f77c-4c67-80cf-4ee52bea3ee8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:38.164980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de01bc-f77c-4c67-80cf-4ee52bea3ee8 None admin] RBAC: Adding query filter params (interface=public, service_id=4135715f22f4433ea3bd0dc6a498586b, region_id=RegionOne) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:38.164980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de01bc-f77c-4c67-80cf-4ee52bea3ee8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:38.165453 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de01bc-f77c-4c67-80cf-4ee52bea3ee8 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:38.170997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 505/1002] 10.222.0.59 () {56 vars in 1251 bytes} [Wed Aug 31 19:06:38 2022] GET /identity/v3/endpoints?service_id=4135715f22f4433ea3bd0dc6a498586b&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:38.176567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555422db-4d04-41b7-85f4-7b13290e3d43 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:38.288696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555422db-4d04-41b7-85f4-7b13290e3d43 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:38.293813 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de01bc-f77c-4c67-80cf-4ee52bea3ee8 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:38.300887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:38.324047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0ad97-c324-484f-8e0f-8d76abf566ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:38.324651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0ad97-c324-484f-8e0f-8d76abf566ab None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:38.324925 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0ad97-c324-484f-8e0f-8d76abf566ab None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:38.325192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0ad97-c324-484f-8e0f-8d76abf566ab None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:38.325837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d0ad97-c324-484f-8e0f-8d76abf566ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:38.327233 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d0ad97-c324-484f-8e0f-8d76abf566ab None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:38.334228 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 498/1003] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 31 19:06:38 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:06:38.466000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de01bc-f77c-4c67-80cf-4ee52bea3ee8 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:38.504244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97935e6b-2f88-434a-bdf3-be01c9c97693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:38.504880 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97935e6b-2f88-434a-bdf3-be01c9c97693 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:38.505150 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97935e6b-2f88-434a-bdf3-be01c9c97693 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:38.505426 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97935e6b-2f88-434a-bdf3-be01c9c97693 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:38.506006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97935e6b-2f88-434a-bdf3-be01c9c97693 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:38.507414 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97935e6b-2f88-434a-bdf3-be01c9c97693 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:38.516436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-97935e6b-2f88-434a-bdf3-be01c9c97693 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:06:38.516436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:38.516436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:38.516436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:38.516436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:38.516436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:38.516436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:38.516436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:38.516436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:06:38.516436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:38.516436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:38.516436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:38.516436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:38.516436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:38.516436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:38.516436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:38.516436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:38.518078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:38.518078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:38.518078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:38.518078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:38.518078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:38.518078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:38.518078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:38.518078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:38.518078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:38.518078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:38.518078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:38.518078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:38.518078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:38.518078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:38.518078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:38.518078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:38.519049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:38.519049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:38.519049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:38.519049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:38.519049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:38.519049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:38.519049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:38.519049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:38.519049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:38.519049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:38.519049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:38.519049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:38.519049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:06:38.519049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:38.519049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 506/1004] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 31 19:06:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:38.524329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0ad97-c324-484f-8e0f-8d76abf566ab None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:38.631924 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0ad97-c324-484f-8e0f-8d76abf566ab None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:38.673717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d98c8-af7d-4d75-be1c-252c0060ec3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:38.674333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d98c8-af7d-4d75-be1c-252c0060ec3b None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:38.674613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d98c8-af7d-4d75-be1c-252c0060ec3b None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:38.674901 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d98c8-af7d-4d75-be1c-252c0060ec3b None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:38.675549 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d98c8-af7d-4d75-be1c-252c0060ec3b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:38.675852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d98c8-af7d-4d75-be1c-252c0060ec3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:38.677217 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d98c8-af7d-4d75-be1c-252c0060ec3b None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:38.687295 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 499/1005] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 31 19:06:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:38.688259 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97935e6b-2f88-434a-bdf3-be01c9c97693 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:38.740602 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:38.794584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97935e6b-2f88-434a-bdf3-be01c9c97693 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:38.830016 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9459033-c874-493b-86de-4893c11a4394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:38.830765 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9459033-c874-493b-86de-4893c11a4394 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:38.831046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9459033-c874-493b-86de-4893c11a4394 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:38.831286 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9459033-c874-493b-86de-4893c11a4394 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:38.831833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9459033-c874-493b-86de-4893c11a4394 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:38.832142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9459033-c874-493b-86de-4893c11a4394 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:38.833517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9459033-c874-493b-86de-4893c11a4394 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:38.843356 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 507/1006] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 31 19:06:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:38.844378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d98c8-af7d-4d75-be1c-252c0060ec3b None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:38.886497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:38.951507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d98c8-af7d-4d75-be1c-252c0060ec3b None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:38.987489 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebfabf-2d1b-4c75-8ba0-246299432d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:38.988110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebfabf-2d1b-4c75-8ba0-246299432d77 None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:38.988391 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebfabf-2d1b-4c75-8ba0-246299432d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:38.988665 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebfabf-2d1b-4c75-8ba0-246299432d77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:38.989350 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebfabf-2d1b-4c75-8ba0-246299432d77 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:38.990711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebfabf-2d1b-4c75-8ba0-246299432d77 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:39.055272 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 500/1007] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:06:38 2022] POST /identity/v3/endpoints => generated 325 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:39.879253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed506d-8a33-4b33-aed6-20800e07d8e6 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:39.879816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed506d-8a33-4b33-aed6-20800e07d8e6 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:39.879816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed506d-8a33-4b33-aed6-20800e07d8e6 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:39.881179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 508/1008] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:39.886308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea4af3-2402-4490-ba32-833621bde61b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:39.886853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea4af3-2402-4490-ba32-833621bde61b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:39.886853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea4af3-2402-4490-ba32-833621bde61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:39.905869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-4bea4af3-2402-4490-ba32-833621bde61b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:39.907647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4bea4af3-2402-4490-ba32-833621bde61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:39.986533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 501/1009] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:40.255786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed506d-8a33-4b33-aed6-20800e07d8e6 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:40.260396 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b0ce9-838c-4da9-946e-bbe0ecb21ab2 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:40.260689 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b0ce9-838c-4da9-946e-bbe0ecb21ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:40.260975 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b0ce9-838c-4da9-946e-bbe0ecb21ab2 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:40.262482 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 502/1010] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:40.267905 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bf4b8-621c-4c97-b16d-d7db5d28fd0d None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:40.271942 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bf4b8-621c-4c97-b16d-d7db5d28fd0d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:40.272252 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bf4b8-621c-4c97-b16d-d7db5d28fd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:40.292458 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-fc3bf4b8-621c-4c97-b16d-d7db5d28fd0d None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:40.293691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-fc3bf4b8-621c-4c97-b16d-d7db5d28fd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:40.314647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:40.368390 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 503/1011] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:40 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:40.377889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed506d-8a33-4b33-aed6-20800e07d8e6 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:40.392799 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:40.414133 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268298d0-e3f0-4cff-ac50-3a619c97875b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:40.414755 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268298d0-e3f0-4cff-ac50-3a619c97875b None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:40.415074 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268298d0-e3f0-4cff-ac50-3a619c97875b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:40.415409 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268298d0-e3f0-4cff-ac50-3a619c97875b None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:40.416078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268298d0-e3f0-4cff-ac50-3a619c97875b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:40.417443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268298d0-e3f0-4cff-ac50-3a619c97875b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:40.472775 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 509/1012] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 31 19:06:40 2022] POST /identity/v3/services => generated 242 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:40.636061 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bf4b8-621c-4c97-b16d-d7db5d28fd0d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:40.744288 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bf4b8-621c-4c97-b16d-d7db5d28fd0d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:40.786815 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e1d9e-de18-4846-b739-566f42e9b688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:40.787488 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e1d9e-de18-4846-b739-566f42e9b688 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:40.787771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e1d9e-de18-4846-b739-566f42e9b688 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:40.788047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e1d9e-de18-4846-b739-566f42e9b688 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:40.788629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e1d9e-de18-4846-b739-566f42e9b688 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:40.790003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e1d9e-de18-4846-b739-566f42e9b688 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:40.799974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-1c4e1d9e-de18-4846-b739-566f42e9b688 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:06:40.799974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:40.799974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:40.799974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:40.799974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:40.799974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:40.799974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:40.799974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:40.799974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:06:40.799974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:40.799974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:40.799974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:40.799974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:40.799974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:40.799974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:40.799974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:40.799974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:40.801106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:40.801106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:40.801106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:40.801106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:40.801106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:40.801106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:40.801106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:40.801106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:40.801106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:40.801106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:40.801106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:40.801106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:40.801106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:40.801106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:40.801106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:40.801106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:40.802014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:40.802014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:40.802014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:40.802014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:40.802014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:40.802014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:40.802014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:40.802014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:40.802014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:40.802014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:40.802014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:40.802014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:40.802014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:06:40.802014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:40.802014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 504/1013] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:40.808741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268298d0-e3f0-4cff-ac50-3a619c97875b None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:40.919963 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268298d0-e3f0-4cff-ac50-3a619c97875b None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:40.955939 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f81cd1-298f-418e-9191-f26dcfa55be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:40.956558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f81cd1-298f-418e-9191-f26dcfa55be7 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:40.956840 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f81cd1-298f-418e-9191-f26dcfa55be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:40.957115 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f81cd1-298f-418e-9191-f26dcfa55be7 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:40.957753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f81cd1-298f-418e-9191-f26dcfa55be7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:40.958065 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f81cd1-298f-418e-9191-f26dcfa55be7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:40.959477 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f81cd1-298f-418e-9191-f26dcfa55be7 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:40.969484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 510/1014] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 31 19:06:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:40.971641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e1d9e-de18-4846-b739-566f42e9b688 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:40.990123 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:41.001448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef5340-e8a3-455f-bb6c-a0466943e1c3 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:41.011130 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef5340-e8a3-455f-bb6c-a0466943e1c3 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:41.011130 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef5340-e8a3-455f-bb6c-a0466943e1c3 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:41.011130 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 511/1015] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:41.011130 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60086d0-b8ca-4cd0-b757-a581ef7773a8 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:41.011130 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60086d0-b8ca-4cd0-b757-a581ef7773a8 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:41.011130 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60086d0-b8ca-4cd0-b757-a581ef7773a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:41.027926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d60086d0-b8ca-4cd0-b757-a581ef7773a8 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:41.029169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d60086d0-b8ca-4cd0-b757-a581ef7773a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:41.061456 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:41.087183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e1d9e-de18-4846-b739-566f42e9b688 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:41.102273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 512/1016] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:41 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:41.127659 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bedacc-a637-4d67-b07d-05a612a9f90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:41.128273 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bedacc-a637-4d67-b07d-05a612a9f90e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:41.128557 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bedacc-a637-4d67-b07d-05a612a9f90e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:41.128833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bedacc-a637-4d67-b07d-05a612a9f90e None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:41.129465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bedacc-a637-4d67-b07d-05a612a9f90e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:41.129771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bedacc-a637-4d67-b07d-05a612a9f90e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:41.131159 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bedacc-a637-4d67-b07d-05a612a9f90e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:41.136654 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 505/1017] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 31 19:06:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:41.143094 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60086d0-b8ca-4cd0-b757-a581ef7773a8 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:41.245400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60086d0-b8ca-4cd0-b757-a581ef7773a8 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:41.284718 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579b68b-84ff-4379-90eb-e18174207b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:41.284903 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579b68b-84ff-4379-90eb-e18174207b3f None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:41.285174 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579b68b-84ff-4379-90eb-e18174207b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:41.285427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579b68b-84ff-4379-90eb-e18174207b3f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:41.286200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579b68b-84ff-4379-90eb-e18174207b3f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:41.287692 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579b68b-84ff-4379-90eb-e18174207b3f None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:41.373401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bedacc-a637-4d67-b07d-05a612a9f90e None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:41.419508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 513/1018] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:06:41 2022] POST /identity/v3/endpoints => generated 334 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:41.464266 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:41.489483 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bedacc-a637-4d67-b07d-05a612a9f90e None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:41.538996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599b65b-a86b-41d8-94cc-85a083158c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:41.539568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599b65b-a86b-41d8-94cc-85a083158c3a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:41.539845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599b65b-a86b-41d8-94cc-85a083158c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:41.540121 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599b65b-a86b-41d8-94cc-85a083158c3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:41.541299 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599b65b-a86b-41d8-94cc-85a083158c3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:41.542767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599b65b-a86b-41d8-94cc-85a083158c3a None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:41.546913 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 506/1019] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:41.550908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579b68b-84ff-4379-90eb-e18174207b3f None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:41.579701 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:41.664639 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579b68b-84ff-4379-90eb-e18174207b3f None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:41.700070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83465770-1cde-489f-98b0-4894467614b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:41.700766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83465770-1cde-489f-98b0-4894467614b0 None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:41.701042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83465770-1cde-489f-98b0-4894467614b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:41.701319 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83465770-1cde-489f-98b0-4894467614b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:41.701988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83465770-1cde-489f-98b0-4894467614b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:41.703494 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83465770-1cde-489f-98b0-4894467614b0 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:41.809292 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 514/1020] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 31 19:06:41 2022] POST /identity/v3/projects => generated 311 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:06:42.478926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3762f52-51fb-4569-b7b2-2131c92bec31 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:42.478926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3762f52-51fb-4569-b7b2-2131c92bec31 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:42.478926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3762f52-51fb-4569-b7b2-2131c92bec31 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:42.482918 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 507/1021] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:42.485447 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90028e8a-6c83-4ce8-81b6-3d2b93c724f4 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:42.486150 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90028e8a-6c83-4ce8-81b6-3d2b93c724f4 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:42.486429 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90028e8a-6c83-4ce8-81b6-3d2b93c724f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:42.504450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-90028e8a-6c83-4ce8-81b6-3d2b93c724f4 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:42.505628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-90028e8a-6c83-4ce8-81b6-3d2b93c724f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:42.574569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 515/1022] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:42.847500 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3762f52-51fb-4569-b7b2-2131c92bec31 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:42.959920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3762f52-51fb-4569-b7b2-2131c92bec31 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:42.994826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359db556-9ee2-4d27-9bbf-7d627328ee6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:42.995490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359db556-9ee2-4d27-9bbf-7d627328ee6d None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:42.995773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359db556-9ee2-4d27-9bbf-7d627328ee6d None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:42.996050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359db556-9ee2-4d27-9bbf-7d627328ee6d None admin] PATH_INFO: `/v3/services/image` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:42.996634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359db556-9ee2-4d27-9bbf-7d627328ee6d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:42.997996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359db556-9ee2-4d27-9bbf-7d627328ee6d None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:43.007004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-359db556-9ee2-4d27-9bbf-7d627328ee6d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:06:43.007004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:43.007004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:43.007004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:43.007004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:43.007004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:43.007004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:43.007004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:43.007004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:06:43.007004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:43.007004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:43.007004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:43.007004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:43.007004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:43.007004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:43.007004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:43.007004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:43.008021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:43.008021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:43.008021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:43.008021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:43.008021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:43.008021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:43.008021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:43.008021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:43.008021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:43.008021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:43.008021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:43.008021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:43.008021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:43.008021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:43.008021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:43.008021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:43.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:43.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:43.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:43.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:43.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:43.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:43.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:43.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:43.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:43.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:43.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:43.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:43.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:06:43.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:43.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 508/1023] 10.222.0.59 () {56 vars in 1105 bytes} [Wed Aug 31 19:06:42 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:43.014311 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90028e8a-6c83-4ce8-81b6-3d2b93c724f4 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:43.023607 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:43.118620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90028e8a-6c83-4ce8-81b6-3d2b93c724f4 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:43.153240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2f2f1-6789-466f-b508-0394860a11bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:43.153855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2f2f1-6789-466f-b508-0394860a11bf None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:43.154137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2f2f1-6789-466f-b508-0394860a11bf None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:43.154410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2f2f1-6789-466f-b508-0394860a11bf None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:43.155073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2f2f1-6789-466f-b508-0394860a11bf None admin] RBAC: Adding query filter params (name=image) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:43.155381 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2f2f1-6789-466f-b508-0394860a11bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:43.156751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2f2f1-6789-466f-b508-0394860a11bf None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:43.166955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 516/1024] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:43.167989 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359db556-9ee2-4d27-9bbf-7d627328ee6d None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:43.177522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:43.277131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359db556-9ee2-4d27-9bbf-7d627328ee6d None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:43.316452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9df64f-efbf-443e-a1ee-79980228757b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:43.317172 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9df64f-efbf-443e-a1ee-79980228757b None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:43.317458 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9df64f-efbf-443e-a1ee-79980228757b None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:43.317734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9df64f-efbf-443e-a1ee-79980228757b None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:43.318367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9df64f-efbf-443e-a1ee-79980228757b None admin] RBAC: Adding query filter params (type=image) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:43.318673 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9df64f-efbf-443e-a1ee-79980228757b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:43.320133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9df64f-efbf-443e-a1ee-79980228757b None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:43.325442 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 509/1025] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:43.332237 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2f2f1-6789-466f-b508-0394860a11bf None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:43.407017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:43.437475 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2f2f1-6789-466f-b508-0394860a11bf None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:43.473141 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1e612-390d-45ab-8698-d0d17ba74fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:43.473748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1e612-390d-45ab-8698-d0d17ba74fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:43.474026 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1e612-390d-45ab-8698-d0d17ba74fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:43.474296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1e612-390d-45ab-8698-d0d17ba74fb8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:43.475023 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b1e612-390d-45ab-8698-d0d17ba74fb8 None admin] RBAC: Adding query filter params (interface=public, service_id=93fb055c76514e6f8180d572493a2d70, region_id=RegionOne) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:43.475331 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b1e612-390d-45ab-8698-d0d17ba74fb8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:43.476690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b1e612-390d-45ab-8698-d0d17ba74fb8 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:43.482259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f535e9c-7fd0-4c29-bed8-6889985def0d None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:43.482726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 517/1026] 10.222.0.59 () {56 vars in 1251 bytes} [Wed Aug 31 19:06:43 2022] GET /identity/v3/endpoints?service_id=93fb055c76514e6f8180d572493a2d70&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:43.482807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f535e9c-7fd0-4c29-bed8-6889985def0d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:43.482807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f535e9c-7fd0-4c29-bed8-6889985def0d None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:43.484076 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 510/1027] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:43.488149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1e612-390d-45ab-8698-d0d17ba74fb8 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:43.488958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a1492-a3a5-4e9e-85fa-4abc38a35f7f None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:43.489250 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a1492-a3a5-4e9e-85fa-4abc38a35f7f None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:43.489530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a1492-a3a5-4e9e-85fa-4abc38a35f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:43.509131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-674a1492-a3a5-4e9e-85fa-4abc38a35f7f None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:43.510338 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-674a1492-a3a5-4e9e-85fa-4abc38a35f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:43.579525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:43.585912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 511/1028] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:43.598707 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1e612-390d-45ab-8698-d0d17ba74fb8 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:43.635140 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ad049-4fd1-471e-a716-6d49cceb3510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:43.635755 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ad049-4fd1-471e-a716-6d49cceb3510 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:43.636034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ad049-4fd1-471e-a716-6d49cceb3510 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:43.636307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ad049-4fd1-471e-a716-6d49cceb3510 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:43.636913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ad049-4fd1-471e-a716-6d49cceb3510 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:43.638297 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ad049-4fd1-471e-a716-6d49cceb3510 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:43.639515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:43.654965 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 518/1029] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 31 19:06:43 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:06:43.763031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d4fb9-924e-43c0-b1e3-29af805a7012 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:43.763209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d4fb9-924e-43c0-b1e3-29af805a7012 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:43.763462 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d4fb9-924e-43c0-b1e3-29af805a7012 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:43.765073 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 512/1030] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:43.771153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9c987-c083-41b4-8c7b-e7a1794be2ab None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:43.771439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9c987-c083-41b4-8c7b-e7a1794be2ab None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:43.771708 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9c987-c083-41b4-8c7b-e7a1794be2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:43.791647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-20b9c987-c083-41b4-8c7b-e7a1794be2ab None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:43.792841 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-20b9c987-c083-41b4-8c7b-e7a1794be2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:43.860321 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d4fb9-924e-43c0-b1e3-29af805a7012 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:43.868843 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 519/1031] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:43.972344 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d4fb9-924e-43c0-b1e3-29af805a7012 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:44.008368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9627b4-c039-4ad8-a243-aaec0ecafc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:44.009003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9627b4-c039-4ad8-a243-aaec0ecafc44 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:44.009286 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9627b4-c039-4ad8-a243-aaec0ecafc44 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:44.009562 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9627b4-c039-4ad8-a243-aaec0ecafc44 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:44.021147 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9627b4-c039-4ad8-a243-aaec0ecafc44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:44.026332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-ab9627b4-c039-4ad8-a243-aaec0ecafc44 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:06:44.026332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:44.026332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:44.026332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:44.026332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:44.026332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:44.026332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:44.026332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:44.026332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:44.026332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:44.026332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:44.026332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:44.026332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:44.026332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:44.026332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:44.026332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:44.027469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:44.027469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:44.027469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:44.027469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:44.027469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:44.027469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:44.027469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:44.027469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:44.027469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:44.027469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:44.027469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:44.027469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:44.027469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:44.027469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:44.027469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:44.027469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:44.032053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:44.032053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:44.032053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:44.032053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:44.032053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:44.032053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:44.032053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:44.032053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:44.032053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:44.032053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:44.032053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:06:44.032053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:44.032053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 513/1032] 10.222.0.59 () {56 vars in 1120 bytes} [Wed Aug 31 19:06:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:44.034783 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9c987-c083-41b4-8c7b-e7a1794be2ab None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:44.141042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9c987-c083-41b4-8c7b-e7a1794be2ab None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:44.143521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9627b4-c039-4ad8-a243-aaec0ecafc44 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:44.174346 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:44.182915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca4ef9-4551-4bb1-ba04-93fd693648c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:44.183532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca4ef9-4551-4bb1-ba04-93fd693648c3 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:44.183812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca4ef9-4551-4bb1-ba04-93fd693648c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:44.184085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca4ef9-4551-4bb1-ba04-93fd693648c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:44.185199 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca4ef9-4551-4bb1-ba04-93fd693648c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:44.185199 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca4ef9-4551-4bb1-ba04-93fd693648c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:44.186412 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca4ef9-4551-4bb1-ba04-93fd693648c3 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:44.200664 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 520/1033] 10.222.0.59 () {56 vars in 1115 bytes} [Wed Aug 31 19:06:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:44.201777 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca4ef9-4551-4bb1-ba04-93fd693648c3 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:44.209951 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:44.254516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9627b4-c039-4ad8-a243-aaec0ecafc44 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:44.294836 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9177f08-e96e-47cb-9e18-2ee9b696b7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:44.295504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9177f08-e96e-47cb-9e18-2ee9b696b7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:44.295786 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9177f08-e96e-47cb-9e18-2ee9b696b7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:44.296060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9177f08-e96e-47cb-9e18-2ee9b696b7d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:44.297220 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9177f08-e96e-47cb-9e18-2ee9b696b7d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:44.298694 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9177f08-e96e-47cb-9e18-2ee9b696b7d4 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:44.300476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 514/1034] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:44.305885 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9177f08-e96e-47cb-9e18-2ee9b696b7d4 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:44.322677 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca4ef9-4551-4bb1-ba04-93fd693648c3 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:44.362363 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a7244-067d-48f6-8a55-0aa17e660737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:44.362683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a7244-067d-48f6-8a55-0aa17e660737 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:44.362991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a7244-067d-48f6-8a55-0aa17e660737 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:44.363264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a7244-067d-48f6-8a55-0aa17e660737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:44.374565 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a7244-067d-48f6-8a55-0aa17e660737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:44.376638 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a7244-067d-48f6-8a55-0aa17e660737 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:44.394024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-258a7244-067d-48f6-8a55-0aa17e660737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:44.394024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:44.394024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:44.394024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:44.394024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:44.394024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:44.394024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:44.394024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:44.394024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:44.394024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:44.394024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:44.394024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:44.394024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:44.394024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:44.394024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:44.394024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:44.395103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:44.395103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:44.395103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:44.395103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:44.395103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:44.395103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:44.395103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:44.395103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:44.395103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:44.395103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:44.395103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:44.395103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:44.395103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:44.395103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:44.395103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:44.395103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:44.395103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:44.396310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:44.396310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:44.396310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:44.396310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 521/1035] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:44.400926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a7244-067d-48f6-8a55-0aa17e660737 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:44.424897 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9177f08-e96e-47cb-9e18-2ee9b696b7d4 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:44.453074 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:44.453493 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:44.463496 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89369c6d-af55-4f89-b259-af3893dc79e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:44.465337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89369c6d-af55-4f89-b259-af3893dc79e3 None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:44.465337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89369c6d-af55-4f89-b259-af3893dc79e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:44.465337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89369c6d-af55-4f89-b259-af3893dc79e3 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:44.465337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89369c6d-af55-4f89-b259-af3893dc79e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:44.466715 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89369c6d-af55-4f89-b259-af3893dc79e3 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:44.513641 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a7244-067d-48f6-8a55-0aa17e660737 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:44.548177 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56812ba4-ce8f-47df-9754-ab20f9d5f31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:44.548687 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56812ba4-ce8f-47df-9754-ab20f9d5f31d None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:44.549058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56812ba4-ce8f-47df-9754-ab20f9d5f31d None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:44.549231 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56812ba4-ce8f-47df-9754-ab20f9d5f31d None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:44.549866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56812ba4-ce8f-47df-9754-ab20f9d5f31d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:44.550171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56812ba4-ce8f-47df-9754-ab20f9d5f31d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:44.551576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56812ba4-ce8f-47df-9754-ab20f9d5f31d None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:44.563797 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 522/1036] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:44.569595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56812ba4-ce8f-47df-9754-ab20f9d5f31d None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:44.597494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 515/1037] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:06:44 2022] POST /identity/v3/users => generated 285 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:06:44.709034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56812ba4-ce8f-47df-9754-ab20f9d5f31d None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:44.710921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:44.757367 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899106d-8637-4355-b83d-407fbc6abeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:44.758026 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899106d-8637-4355-b83d-407fbc6abeaf None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:44.758317 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899106d-8637-4355-b83d-407fbc6abeaf None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:44.758590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899106d-8637-4355-b83d-407fbc6abeaf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:44.767871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899106d-8637-4355-b83d-407fbc6abeaf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:44.769340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899106d-8637-4355-b83d-407fbc6abeaf None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:44.781731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-4899106d-8637-4355-b83d-407fbc6abeaf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:06:44.781731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:44.781731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:44.781731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:44.781731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:44.781731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:44.781731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:44.781731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:44.781731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:44.781731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:44.781731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:44.781731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:44.781731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:06:44.781731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:06:44.781731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:06:44.781731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:44.783652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:44.784924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:44.784924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:44.784924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:06:44.784924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:06:44.784924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:06:44.784924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:06:44.784924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:06:44.784924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:06:44.784924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:06:44.784924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:44.784924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 523/1038] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 31 19:06:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:44.794282 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89369c6d-af55-4f89-b259-af3893dc79e3 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:44.917370 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89369c6d-af55-4f89-b259-af3893dc79e3 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:44.958093 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db60e39-5f1c-4072-8cc0-8021f791f430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:44.958720 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db60e39-5f1c-4072-8cc0-8021f791f430 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:44.959044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db60e39-5f1c-4072-8cc0-8021f791f430 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:44.959321 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db60e39-5f1c-4072-8cc0-8021f791f430 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:44.960055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db60e39-5f1c-4072-8cc0-8021f791f430 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:44.960365 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db60e39-5f1c-4072-8cc0-8021f791f430 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:44.961788 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db60e39-5f1c-4072-8cc0-8021f791f430 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:44.981423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 516/1039] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 31 19:06:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:44.988419 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899106d-8637-4355-b83d-407fbc6abeaf None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:45.091757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899106d-8637-4355-b83d-407fbc6abeaf None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:45.128540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960afe7-c342-4d2d-ac77-875a33529796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:45.129159 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960afe7-c342-4d2d-ac77-875a33529796 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:45.129443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960afe7-c342-4d2d-ac77-875a33529796 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:45.129720 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960afe7-c342-4d2d-ac77-875a33529796 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:45.162728 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:45.202585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960afe7-c342-4d2d-ac77-875a33529796 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:45.204050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960afe7-c342-4d2d-ac77-875a33529796 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:45.327276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 524/1040] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:06:44 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:06:45.328498 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db60e39-5f1c-4072-8cc0-8021f791f430 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:45.376366 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:45.439132 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db60e39-5f1c-4072-8cc0-8021f791f430 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:45.480478 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f29f9-0524-482f-a03b-a2aff978b484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:45.480997 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f29f9-0524-482f-a03b-a2aff978b484 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:45.481278 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f29f9-0524-482f-a03b-a2aff978b484 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:45.481554 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f29f9-0524-482f-a03b-a2aff978b484 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:45.492862 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f29f9-0524-482f-a03b-a2aff978b484 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:45.494316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f29f9-0524-482f-a03b-a2aff978b484 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:45.509807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-957f29f9-0524-482f-a03b-a2aff978b484 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:45.509807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:45.509807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:45.509807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:45.509807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:45.509807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:45.509807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:45.509807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:45.509807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:45.509807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:45.509807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:45.509807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:45.509807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:45.509807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:45.509807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:45.509807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:45.516859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:45.516859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:45.516859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:45.516859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:45.516859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:45.516859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:45.516859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:45.516859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:45.516859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:45.516859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:45.516859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:45.516859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:45.516859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:45.516859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:45.516859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:45.516859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:45.516859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:45.517894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:45.517894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:45.517894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:45.517894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 517/1041] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:45.518140 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960afe7-c342-4d2d-ac77-875a33529796 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:45.588281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:45.601071 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d1a3c-ee3b-48ba-9b21-6cdf099b82b3 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:45.601941 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d1a3c-ee3b-48ba-9b21-6cdf099b82b3 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:45.601941 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d1a3c-ee3b-48ba-9b21-6cdf099b82b3 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:45.603268 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 518/1042] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:45.608839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe02ac5-0416-4274-b478-9ac900b3b211 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:45.609390 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe02ac5-0416-4274-b478-9ac900b3b211 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:45.609667 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe02ac5-0416-4274-b478-9ac900b3b211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:45.632028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-1fe02ac5-0416-4274-b478-9ac900b3b211 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:45.633252 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-1fe02ac5-0416-4274-b478-9ac900b3b211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:45.635422 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960afe7-c342-4d2d-ac77-875a33529796 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:45.682408 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f91087-8675-4cdb-849d-62ae97f1be32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:45.683487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f91087-8675-4cdb-849d-62ae97f1be32 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:45.684043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f91087-8675-4cdb-849d-62ae97f1be32 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:45.684427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f91087-8675-4cdb-849d-62ae97f1be32 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:45.685090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f91087-8675-4cdb-849d-62ae97f1be32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:45.685396 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f91087-8675-4cdb-849d-62ae97f1be32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:45.687540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f91087-8675-4cdb-849d-62ae97f1be32 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:45.706564 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 525/1043] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:45.707541 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f91087-8675-4cdb-849d-62ae97f1be32 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:45.718911 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 519/1044] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:45.818106 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f91087-8675-4cdb-849d-62ae97f1be32 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:45.842002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:45.854779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f389120-7bc2-424f-9cc7-3cc182bfa50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:45.855618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f389120-7bc2-424f-9cc7-3cc182bfa50c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:45.856067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f389120-7bc2-424f-9cc7-3cc182bfa50c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:45.856483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f389120-7bc2-424f-9cc7-3cc182bfa50c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:45.864557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f389120-7bc2-424f-9cc7-3cc182bfa50c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:45.866132 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f389120-7bc2-424f-9cc7-3cc182bfa50c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:45.877806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-0f389120-7bc2-424f-9cc7-3cc182bfa50c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:45.877806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:45.877806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:45.877806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:45.877806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:45.877806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:45.877806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:45.877806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:45.877806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:45.877806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:45.877806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:45.877806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:45.877806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:06:45.877806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:06:45.877806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:06:45.878931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:06:45.878931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:45.878931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:45.878931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:45.878931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:45.878931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:45.878931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:45.878931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:45.878931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:45.878931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:45.878931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:45.878931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:45.878931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:45.878931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:45.878931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:45.878931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:45.878931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:45.880071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:06:45.880071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:06:45.880071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:06:45.880071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:06:45.880071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:06:45.880071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:06:45.880071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:45.880071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:45.881808 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 526/1045] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:45.887609 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe02ac5-0416-4274-b478-9ac900b3b211 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:45.913621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:45.988008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f389120-7bc2-424f-9cc7-3cc182bfa50c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:45.995341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe02ac5-0416-4274-b478-9ac900b3b211 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:46.033783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3de04e-9157-45c4-815b-e0d5b1f89d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:46.034498 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3de04e-9157-45c4-815b-e0d5b1f89d36 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:46.034680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3de04e-9157-45c4-815b-e0d5b1f89d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:46.034980 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3de04e-9157-45c4-815b-e0d5b1f89d36 None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:46.035649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3de04e-9157-45c4-815b-e0d5b1f89d36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:46.035954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3de04e-9157-45c4-815b-e0d5b1f89d36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:46.037639 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3de04e-9157-45c4-815b-e0d5b1f89d36 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:46.050968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 520/1046] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:06:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:46.056727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3de04e-9157-45c4-815b-e0d5b1f89d36 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:46.099431 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f389120-7bc2-424f-9cc7-3cc182bfa50c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:46.137407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a59200-7de3-4ca6-9533-24dd90d15ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:46.138403 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a59200-7de3-4ca6-9533-24dd90d15ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:46.138917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a59200-7de3-4ca6-9533-24dd90d15ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:46.139181 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a59200-7de3-4ca6-9533-24dd90d15ae2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:46.139774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a59200-7de3-4ca6-9533-24dd90d15ae2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:46.141640 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a59200-7de3-4ca6-9533-24dd90d15ae2 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:46.148293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:46.151279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-67a59200-7de3-4ca6-9533-24dd90d15ae2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:06:46.151279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:46.151279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:06:46.151279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:06:46.151279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:06:46.151279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:06:46.151279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:06:46.151279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:46.151279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:06:46.151279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:46.151279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:46.151279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:46.151279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:46.151279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:46.151279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:46.151279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:46.151279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:46.152296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:46.152296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:46.152296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:46.152296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:46.152296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:46.152296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:46.152296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:06:46.152296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:06:46.152296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:06:46.152296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:06:46.152296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:46.152296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:46.152296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:46.152296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:46.152296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:46.152296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:46.153302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:46.153302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:46.153302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:46.153302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:46.153302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:46.153302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:46.153302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:46.153302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:46.153302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:46.153302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:46.153302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:06:46.153302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:06:46.153302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:06:46.153302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:46.153302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 527/1047] 10.222.0.59 () {56 vars in 1105 bytes} [Wed Aug 31 19:06:45 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:46.158763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a59200-7de3-4ca6-9533-24dd90d15ae2 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:46.167976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3de04e-9157-45c4-815b-e0d5b1f89d36 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:46.206206 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbef1c-7c16-49ed-a088-290d0bd6ecb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:46.206862 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbef1c-7c16-49ed-a088-290d0bd6ecb6 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:46.207169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbef1c-7c16-49ed-a088-290d0bd6ecb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:46.207442 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbef1c-7c16-49ed-a088-290d0bd6ecb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:46.208506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbef1c-7c16-49ed-a088-290d0bd6ecb6 None admin] RBAC: Adding query filter params (role.id=e22b60b1c78d45b29c81bcce9064de49, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=d461422ce98442bcbe6b254a8dad5e8a) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:46.208506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbef1c-7c16-49ed-a088-290d0bd6ecb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:46.209927 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbef1c-7c16-49ed-a088-290d0bd6ecb6 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:46.214572 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:46.216169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 521/1048] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:06:46 2022] GET /identity/v3/role_assignments?user.id=d461422ce98442bcbe6b254a8dad5e8a&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=e22b60b1c78d45b29c81bcce9064de49 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:46.289161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a59200-7de3-4ca6-9533-24dd90d15ae2 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:46.332860 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34c1d1-b65e-4f15-9a67-9393afe61f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:46.333495 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34c1d1-b65e-4f15-9a67-9393afe61f25 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:46.333828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34c1d1-b65e-4f15-9a67-9393afe61f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:46.334105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34c1d1-b65e-4f15-9a67-9393afe61f25 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:46.334759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34c1d1-b65e-4f15-9a67-9393afe61f25 None admin] RBAC: Adding query filter params (name=image) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:46.335098 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34c1d1-b65e-4f15-9a67-9393afe61f25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:46.336490 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34c1d1-b65e-4f15-9a67-9393afe61f25 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:46.347698 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 528/1049] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:46.349227 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbef1c-7c16-49ed-a088-290d0bd6ecb6 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:46.455446 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbef1c-7c16-49ed-a088-290d0bd6ecb6 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:46.490596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83fd95-0f98-46b5-a07b-8e88d49899c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:46.491241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83fd95-0f98-46b5-a07b-8e88d49899c6 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:46.491520 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83fd95-0f98-46b5-a07b-8e88d49899c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:46.491790 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83fd95-0f98-46b5-a07b-8e88d49899c6 None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:46.492417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83fd95-0f98-46b5-a07b-8e88d49899c6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:46.492715 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83fd95-0f98-46b5-a07b-8e88d49899c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:46.494064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83fd95-0f98-46b5-a07b-8e88d49899c6 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:46.504243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 522/1050] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:46.505444 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34c1d1-b65e-4f15-9a67-9393afe61f25 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:46.614713 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34c1d1-b65e-4f15-9a67-9393afe61f25 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:46.625438 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:46.651790 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c933800-80e2-4a9e-b5e5-e0ab8db5dd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:46.652408 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c933800-80e2-4a9e-b5e5-e0ab8db5dd57 None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:46.652687 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c933800-80e2-4a9e-b5e5-e0ab8db5dd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:46.652960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c933800-80e2-4a9e-b5e5-e0ab8db5dd57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:46.653768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c933800-80e2-4a9e-b5e5-e0ab8db5dd57 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:46.655035 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c933800-80e2-4a9e-b5e5-e0ab8db5dd57 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:46.674677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf4713-d934-4ede-be9d-20b6dc5fd6a6 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:46.675460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf4713-d934-4ede-be9d-20b6dc5fd6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:46.675460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf4713-d934-4ede-be9d-20b6dc5fd6a6 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:46.677766 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 523/1051] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:46.681882 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c62943-4d97-4847-8e1a-cb6be8ffa23e None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:46.682168 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c62943-4d97-4847-8e1a-cb6be8ffa23e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:46.682446 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c62943-4d97-4847-8e1a-cb6be8ffa23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:46.701983 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-01c62943-4d97-4847-8e1a-cb6be8ffa23e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:46.703238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-01c62943-4d97-4847-8e1a-cb6be8ffa23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:46.711956 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:46.766899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 529/1052] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:06:46 2022] POST /identity/v3/endpoints => generated 327 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:46.785576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 524/1053] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:47.059658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c933800-80e2-4a9e-b5e5-e0ab8db5dd57 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:47.164126 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c933800-80e2-4a9e-b5e5-e0ab8db5dd57 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:47.198774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece40531-0348-40be-a6b0-d5055a246991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:47.199440 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece40531-0348-40be-a6b0-d5055a246991 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:47.199721 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece40531-0348-40be-a6b0-d5055a246991 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:47.199998 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece40531-0348-40be-a6b0-d5055a246991 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:47.211401 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece40531-0348-40be-a6b0-d5055a246991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:47.216227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-ece40531-0348-40be-a6b0-d5055a246991 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:47.216227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:47.216227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:47.216227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:47.216227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:47.216227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:47.216227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:47.216227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:47.216227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:47.216227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:47.216227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:47.216227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:47.216227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:47.216227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:47.216227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:47.216227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:47.217248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:47.217248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:47.217248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:47.217248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:47.217248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:47.217248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:47.217248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:47.217248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:47.217248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:47.217248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:47.217248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:47.217248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:47.217248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:47.217248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:47.217248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:47.217248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:47.218123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:47.218123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:47.218123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:47.218123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:47.218123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:47.218123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:47.218123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:47.218123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:47.218123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:47.218123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:47.218123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:47.218123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:47.222390 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 530/1054] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:06:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:47.223850 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c62943-4d97-4847-8e1a-cb6be8ffa23e None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:47.336226 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c62943-4d97-4847-8e1a-cb6be8ffa23e None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:47.371337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e18ab-4c29-4c54-a5a3-5d4183ad5ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:47.372151 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e18ab-4c29-4c54-a5a3-5d4183ad5ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:47.372307 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e18ab-4c29-4c54-a5a3-5d4183ad5ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:47.372481 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e18ab-4c29-4c54-a5a3-5d4183ad5ec0 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:47.373143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e18ab-4c29-4c54-a5a3-5d4183ad5ec0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:47.373448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e18ab-4c29-4c54-a5a3-5d4183ad5ec0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:47.374803 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e18ab-4c29-4c54-a5a3-5d4183ad5ec0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:47.380971 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:47.389638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 525/1055] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:06:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:47.391182 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece40531-0348-40be-a6b0-d5055a246991 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:47.423435 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:47.511940 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece40531-0348-40be-a6b0-d5055a246991 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:47.548619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644607c-2601-4060-9a7b-591439c66f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:47.549264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644607c-2601-4060-9a7b-591439c66f4c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:47.549549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644607c-2601-4060-9a7b-591439c66f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:47.549819 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644607c-2601-4060-9a7b-591439c66f4c None admin] PATH_INFO: `/v3/users/73f5fcea28044a128b957bd075db3de0` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:47.564442 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644607c-2601-4060-9a7b-591439c66f4c None admin] RBAC: Authorizing `identity:get_user(user_id=73f5fcea28044a128b957bd075db3de0)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:47.565915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644607c-2601-4060-9a7b-591439c66f4c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:47.572504 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 531/1056] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:06:47 2022] GET /identity/v3/users/73f5fcea28044a128b957bd075db3de0 => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:47.574125 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e18ab-4c29-4c54-a5a3-5d4183ad5ec0 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:47.684496 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e18ab-4c29-4c54-a5a3-5d4183ad5ec0 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:47.730601 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850819b0-61df-4fb8-8dfb-45d318c20f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:47.731389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850819b0-61df-4fb8-8dfb-45d318c20f41 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:47.731389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850819b0-61df-4fb8-8dfb-45d318c20f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:47.731573 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850819b0-61df-4fb8-8dfb-45d318c20f41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:47.804572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:47.807557 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850819b0-61df-4fb8-8dfb-45d318c20f41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:47.809112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850819b0-61df-4fb8-8dfb-45d318c20f41 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:47.921411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 526/1057] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:06:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:06:47.923105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644607c-2601-4060-9a7b-591439c66f4c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:48.010537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:48.027686 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644607c-2601-4060-9a7b-591439c66f4c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:48.062308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4d705-d2ff-499a-af35-cf33aca3a6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:48.063651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4d705-d2ff-499a-af35-cf33aca3a6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:48.063933 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4d705-d2ff-499a-af35-cf33aca3a6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:48.064215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4d705-d2ff-499a-af35-cf33aca3a6a5 None admin] PATH_INFO: `/v3/projects/dc014f087b5944b8843f1932af169153` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:48.065335 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4d705-d2ff-499a-af35-cf33aca3a6a5 None admin] RBAC: Authorizing `identity:get_project(project_id=dc014f087b5944b8843f1932af169153)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:48.066764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4d705-d2ff-499a-af35-cf33aca3a6a5 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:48.073362 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 532/1058] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:06:47 2022] GET /identity/v3/projects/dc014f087b5944b8843f1932af169153 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:48.074489 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850819b0-61df-4fb8-8dfb-45d318c20f41 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:48.163595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc2c54-0b9b-433c-868e-922106870358 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:48.163919 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc2c54-0b9b-433c-868e-922106870358 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:48.164162 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc2c54-0b9b-433c-868e-922106870358 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:48.165727 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 533/1059] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:48.171164 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818516d-8e27-46ed-80c4-90c6a5dcc2da None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:48.171447 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818516d-8e27-46ed-80c4-90c6a5dcc2da None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:48.171717 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818516d-8e27-46ed-80c4-90c6a5dcc2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:48.179712 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:48.183000 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850819b0-61df-4fb8-8dfb-45d318c20f41 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:48.191167 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-f818516d-8e27-46ed-80c4-90c6a5dcc2da None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:48.192348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f818516d-8e27-46ed-80c4-90c6a5dcc2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:48.220944 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c2d82-8c75-49be-98c3-fe9d7af42ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:48.221555 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c2d82-8c75-49be-98c3-fe9d7af42ded None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:48.221833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c2d82-8c75-49be-98c3-fe9d7af42ded None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:48.222101 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c2d82-8c75-49be-98c3-fe9d7af42ded None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:48.222824 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c2d82-8c75-49be-98c3-fe9d7af42ded None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=dc014f087b5944b8843f1932af169153, user.id=73f5fcea28044a128b957bd075db3de0) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:48.223454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c2d82-8c75-49be-98c3-fe9d7af42ded None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:48.225173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c2d82-8c75-49be-98c3-fe9d7af42ded None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:48.240672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 527/1060] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:06:48 2022] GET /identity/v3/role_assignments?user.id=73f5fcea28044a128b957bd075db3de0&scope.project.id=dc014f087b5944b8843f1932af169153&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:48.270299 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 534/1061] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:48.541388 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c2d82-8c75-49be-98c3-fe9d7af42ded None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:48.646733 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c2d82-8c75-49be-98c3-fe9d7af42ded None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:48.683770 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4989f-e2ca-49bc-aa9a-3747a17ace15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:48.684407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4989f-e2ca-49bc-aa9a-3747a17ace15 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:48.684679 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4989f-e2ca-49bc-aa9a-3747a17ace15 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:48.685220 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4989f-e2ca-49bc-aa9a-3747a17ace15 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:48.696300 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4989f-e2ca-49bc-aa9a-3747a17ace15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:48.701070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-7cf4989f-e2ca-49bc-aa9a-3747a17ace15 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:06:48.701070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:48.701070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:48.701070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:48.701070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:48.701070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:48.701070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:48.701070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:48.701070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:48.701070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:48.701070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:48.701070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:48.701070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:48.701070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:48.701070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:48.701070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:48.702030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:48.702030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:48.702030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:48.702030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:48.702030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:48.702030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:48.702030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:48.702030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:48.702030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:48.702030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:48.702030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:48.702030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:48.702030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:48.702030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:48.702030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:48.702030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:48.704680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:48.704680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:48.704680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:48.704680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:48.704680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:48.704680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:48.704680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:48.704680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:48.704680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:48.704680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:48.704680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:06:48.704680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:48.704680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 528/1062] 10.222.0.59 () {56 vars in 1120 bytes} [Wed Aug 31 19:06:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:48.709194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818516d-8e27-46ed-80c4-90c6a5dcc2da None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:48.744502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff68f7-100a-423e-9ba3-b34bf61f8e94 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:48.744794 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff68f7-100a-423e-9ba3-b34bf61f8e94 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:48.745063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff68f7-100a-423e-9ba3-b34bf61f8e94 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:48.746425 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 529/1063] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:48.752550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676142f3-f092-4c91-8d80-2fe7754c5571 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:48.752826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676142f3-f092-4c91-8d80-2fe7754c5571 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:48.753157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676142f3-f092-4c91-8d80-2fe7754c5571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:48.772204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-676142f3-f092-4c91-8d80-2fe7754c5571 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:48.773424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-676142f3-f092-4c91-8d80-2fe7754c5571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:48.792954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:48.803440 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:48.823575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818516d-8e27-46ed-80c4-90c6a5dcc2da None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:48.846949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 530/1064] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:48.859802 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be6ecb-2f43-4e4c-8fcd-fcec37db4b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:48.861502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be6ecb-2f43-4e4c-8fcd-fcec37db4b81 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:48.861914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be6ecb-2f43-4e4c-8fcd-fcec37db4b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:48.862420 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be6ecb-2f43-4e4c-8fcd-fcec37db4b81 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:48.863275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be6ecb-2f43-4e4c-8fcd-fcec37db4b81 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:48.863745 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be6ecb-2f43-4e4c-8fcd-fcec37db4b81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:48.865287 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be6ecb-2f43-4e4c-8fcd-fcec37db4b81 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:48.874895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 535/1065] 10.222.0.59 () {56 vars in 1115 bytes} [Wed Aug 31 19:06:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:48.880737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676142f3-f092-4c91-8d80-2fe7754c5571 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:48.985998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676142f3-f092-4c91-8d80-2fe7754c5571 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:49.019623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7bbee-2e90-4266-9374-c5ea280920b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:49.020251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7bbee-2e90-4266-9374-c5ea280920b2 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:49.020531 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7bbee-2e90-4266-9374-c5ea280920b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:49.020802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7bbee-2e90-4266-9374-c5ea280920b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:49.031111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7bbee-2e90-4266-9374-c5ea280920b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:49.033240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7bbee-2e90-4266-9374-c5ea280920b2 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:49.048817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-dbe7bbee-2e90-4266-9374-c5ea280920b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:49.048817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:49.048817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:49.048817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:49.048817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:49.048817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:49.048817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:49.048817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:49.048817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:49.048817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:49.048817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:49.048817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:49.048817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:49.048817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:49.048817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:49.048817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:49.049789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:49.049789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:49.049789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:49.049789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:49.049789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:49.049789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:49.049789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:49.049789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:49.049789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:49.049789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:49.049789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:49.049789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:49.049789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:49.049789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:49.049789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:49.049789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:49.049789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:49.055758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:49.055758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:49.055758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:49.055758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 531/1066] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:49.057290 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be6ecb-2f43-4e4c-8fcd-fcec37db4b81 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:49.113895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7bbee-2e90-4266-9374-c5ea280920b2 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:49.123706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:49.180716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be6ecb-2f43-4e4c-8fcd-fcec37db4b81 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:49.217692 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963043e8-f031-4267-a671-a205342dd6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:49.218613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963043e8-f031-4267-a671-a205342dd6aa None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:49.219155 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963043e8-f031-4267-a671-a205342dd6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:49.219999 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963043e8-f031-4267-a671-a205342dd6aa None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:49.220654 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963043e8-f031-4267-a671-a205342dd6aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:49.220961 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963043e8-f031-4267-a671-a205342dd6aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:49.222339 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963043e8-f031-4267-a671-a205342dd6aa None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:49.226039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7bbee-2e90-4266-9374-c5ea280920b2 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:49.234709 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 536/1067] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:49.240884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963043e8-f031-4267-a671-a205342dd6aa None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:49.245215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:49.267341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f837f-2353-484c-a1fc-1814d109ce7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:49.268057 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f837f-2353-484c-a1fc-1814d109ce7f None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:49.268344 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f837f-2353-484c-a1fc-1814d109ce7f None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:49.272103 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f837f-2353-484c-a1fc-1814d109ce7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:49.355190 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963043e8-f031-4267-a671-a205342dd6aa None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:49.361622 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f837f-2353-484c-a1fc-1814d109ce7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:49.363102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f837f-2353-484c-a1fc-1814d109ce7f None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:49.372428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:49.394240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252645bb-1a8d-4e1b-a9e2-c4daf04d6f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:49.395226 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252645bb-1a8d-4e1b-a9e2-c4daf04d6f21 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:49.395624 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252645bb-1a8d-4e1b-a9e2-c4daf04d6f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:49.395885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252645bb-1a8d-4e1b-a9e2-c4daf04d6f21 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:49.403002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252645bb-1a8d-4e1b-a9e2-c4daf04d6f21 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:49.404437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252645bb-1a8d-4e1b-a9e2-c4daf04d6f21 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:49.416145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-252645bb-1a8d-4e1b-a9e2-c4daf04d6f21 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:06:49.416145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:49.416145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:49.416145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:49.416145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:49.416145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:49.416145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:49.416145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:49.416145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:49.416145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:49.416145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:49.416145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:49.416145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:06:49.416145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:06:49.416145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:06:49.416145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:49.417376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:49.423726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:49.423726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:49.423726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:06:49.423726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:06:49.423726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:06:49.423726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:06:49.423726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:06:49.423726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:06:49.423726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:06:49.423726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:49.423726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 537/1068] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 31 19:06:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:49.423726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252645bb-1a8d-4e1b-a9e2-c4daf04d6f21 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:49.481738 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 532/1069] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:06:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 19:06:49.489665 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:49.491849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f837f-2353-484c-a1fc-1814d109ce7f None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:49.536492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252645bb-1a8d-4e1b-a9e2-c4daf04d6f21 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:49.571870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779679a6-e8ac-4847-842d-2ed3b6ee9694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:49.572632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779679a6-e8ac-4847-842d-2ed3b6ee9694 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:49.573032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779679a6-e8ac-4847-842d-2ed3b6ee9694 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:49.573404 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779679a6-e8ac-4847-842d-2ed3b6ee9694 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:49.574237 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779679a6-e8ac-4847-842d-2ed3b6ee9694 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:49.574641 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779679a6-e8ac-4847-842d-2ed3b6ee9694 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:49.576329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779679a6-e8ac-4847-842d-2ed3b6ee9694 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:49.599532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 538/1070] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 31 19:06:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:49.600585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779679a6-e8ac-4847-842d-2ed3b6ee9694 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:49.605351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f837f-2353-484c-a1fc-1814d109ce7f None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:49.614546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:49.643522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa7491-356c-4555-a19e-90a0e8ea921e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:49.644154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa7491-356c-4555-a19e-90a0e8ea921e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:49.644428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa7491-356c-4555-a19e-90a0e8ea921e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:49.644696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa7491-356c-4555-a19e-90a0e8ea921e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:49.655160 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa7491-356c-4555-a19e-90a0e8ea921e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:49.656605 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa7491-356c-4555-a19e-90a0e8ea921e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:49.672054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-93fa7491-356c-4555-a19e-90a0e8ea921e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:49.672054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:49.672054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:49.672054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:49.672054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:49.672054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:49.672054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:49.672054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:49.672054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:49.672054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:49.672054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:49.672054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:49.672054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:49.672054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:49.672054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:49.672054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:49.674264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:49.674264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:49.674264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:49.674264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:49.674264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:49.674264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:49.674264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:49.674264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:49.674264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:49.674264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:49.674264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:49.674264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:49.674264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:49.674264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:49.674264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:49.674264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:49.674264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:49.675440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:49.675440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:49.675440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:49.679262 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 533/1071] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:49.680110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa7491-356c-4555-a19e-90a0e8ea921e None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:49.712602 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779679a6-e8ac-4847-842d-2ed3b6ee9694 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:49.738098 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:49.749827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed7942-7408-478b-b506-1f0c35683dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:49.751710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed7942-7408-478b-b506-1f0c35683dfc None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:49.751710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed7942-7408-478b-b506-1f0c35683dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:49.751710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed7942-7408-478b-b506-1f0c35683dfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:49.761757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed7942-7408-478b-b506-1f0c35683dfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:49.763243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed7942-7408-478b-b506-1f0c35683dfc None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:49.778020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-e4ed7942-7408-478b-b506-1f0c35683dfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:49.778020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:49.778020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:49.778020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:49.778020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:49.778020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:49.778020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:49.778020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:49.778020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:49.778020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:49.778020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:49.778020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:49.778020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:49.778020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:49.778020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:49.778020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:49.779127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:49.779127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:49.779127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:49.779127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:49.779127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:49.779127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:49.779127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:49.779127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:49.779127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:49.779127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:49.779127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:49.779127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:49.779127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:49.779127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:49.779127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:49.779127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:49.779127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:49.780057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:49.780057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:49.780057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:49.780057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 539/1072] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:49.784889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed7942-7408-478b-b506-1f0c35683dfc None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:49.808379 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa7491-356c-4555-a19e-90a0e8ea921e None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:49.845754 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbbfb7-352a-418f-a512-0fbf116d3db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:49.846366 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbbfb7-352a-418f-a512-0fbf116d3db1 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:49.846643 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbbfb7-352a-418f-a512-0fbf116d3db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:49.846932 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbbfb7-352a-418f-a512-0fbf116d3db1 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:49.847572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbbfb7-352a-418f-a512-0fbf116d3db1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:49.847872 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbbfb7-352a-418f-a512-0fbf116d3db1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:49.849223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbbfb7-352a-418f-a512-0fbf116d3db1 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:49.863544 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 534/1073] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:49.895367 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed7942-7408-478b-b506-1f0c35683dfc None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:49.934450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713edfb4-1394-4b0c-9d9c-81ba83e95c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:49.935116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713edfb4-1394-4b0c-9d9c-81ba83e95c44 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:49.935399 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713edfb4-1394-4b0c-9d9c-81ba83e95c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:49.935787 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713edfb4-1394-4b0c-9d9c-81ba83e95c44 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:49.936345 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713edfb4-1394-4b0c-9d9c-81ba83e95c44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:49.936655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713edfb4-1394-4b0c-9d9c-81ba83e95c44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:49.938090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713edfb4-1394-4b0c-9d9c-81ba83e95c44 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:49.956807 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 540/1074] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:49.957915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbbfb7-352a-418f-a512-0fbf116d3db1 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:49.980213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:50.072841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbbfb7-352a-418f-a512-0fbf116d3db1 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:50.111185 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc6023-bd33-4227-8753-d849940a1cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:50.111831 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc6023-bd33-4227-8753-d849940a1cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:50.112106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc6023-bd33-4227-8753-d849940a1cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:50.112375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc6023-bd33-4227-8753-d849940a1cd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:50.119523 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc6023-bd33-4227-8753-d849940a1cd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:50.120888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc6023-bd33-4227-8753-d849940a1cd2 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:50.133177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-33fc6023-bd33-4227-8753-d849940a1cd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:50.133177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:50.133177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:50.133177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:50.133177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:50.133177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:50.133177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:50.133177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:50.133177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:50.133177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:50.133177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:50.133177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:50.133177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:06:50.133177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:06:50.133177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:06:50.134153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:06:50.134153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:50.134153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:50.134153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:50.134153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:50.134153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:50.134153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:50.134153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:50.134153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:50.134153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:50.134153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:50.134153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:50.134153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:50.134153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:50.134153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:50.134153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:50.134153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:50.135676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:06:50.135676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:06:50.135676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:06:50.135676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:06:50.135676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:06:50.135676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:06:50.135676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:50.135676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:50.135676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 535/1075] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:50.140207 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713edfb4-1394-4b0c-9d9c-81ba83e95c44 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:50.164226 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0320b1-b2df-41cf-88e9-bea4ad4c4df1 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:50.164792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0320b1-b2df-41cf-88e9-bea4ad4c4df1 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:50.165163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0320b1-b2df-41cf-88e9-bea4ad4c4df1 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:50.166571 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 536/1076] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:50.171816 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381ed88-9159-4f96-ac36-0ae5a7e40cf1 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:50.172098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381ed88-9159-4f96-ac36-0ae5a7e40cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:50.172472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381ed88-9159-4f96-ac36-0ae5a7e40cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:50.183071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:50.190843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-8381ed88-9159-4f96-ac36-0ae5a7e40cf1 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:50.192099 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8381ed88-9159-4f96-ac36-0ae5a7e40cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:50.251858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713edfb4-1394-4b0c-9d9c-81ba83e95c44 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:50.260696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 537/1077] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:50.289491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842167dc-f892-4964-b8c1-c1c135ea5712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:50.290100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842167dc-f892-4964-b8c1-c1c135ea5712 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:50.290375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842167dc-f892-4964-b8c1-c1c135ea5712 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:50.290644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842167dc-f892-4964-b8c1-c1c135ea5712 None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:50.291364 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842167dc-f892-4964-b8c1-c1c135ea5712 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:50.291669 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842167dc-f892-4964-b8c1-c1c135ea5712 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:50.293282 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842167dc-f892-4964-b8c1-c1c135ea5712 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:50.304688 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 541/1078] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:06:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:50.311024 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381ed88-9159-4f96-ac36-0ae5a7e40cf1 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:50.322207 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:50.411125 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381ed88-9159-4f96-ac36-0ae5a7e40cf1 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:50.445191 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be6f3f-857d-463e-a5e2-c343d13d63ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:50.445909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be6f3f-857d-463e-a5e2-c343d13d63ed None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:50.446190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be6f3f-857d-463e-a5e2-c343d13d63ed None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:50.446491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be6f3f-857d-463e-a5e2-c343d13d63ed None admin] PATH_INFO: `/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/d461422ce98442bcbe6b254a8dad5e8a/roles/e22b60b1c78d45b29c81bcce9064de49` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:50.448649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be6f3f-857d-463e-a5e2-c343d13d63ed None admin] RBAC: Authorizing `identity:create_grant(project_id=161c90d34bb34925a1aaeb70c7de7eae, user_id=d461422ce98442bcbe6b254a8dad5e8a, role_id=e22b60b1c78d45b29c81bcce9064de49)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:50.450080 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be6f3f-857d-463e-a5e2-c343d13d63ed None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:50.509532 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 538/1079] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:06:50 2022] PUT /identity/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/d461422ce98442bcbe6b254a8dad5e8a/roles/e22b60b1c78d45b29c81bcce9064de49 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:06:50.533613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842167dc-f892-4964-b8c1-c1c135ea5712 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:50.638177 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:50.646446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842167dc-f892-4964-b8c1-c1c135ea5712 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:50.680697 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e986aa-1a0d-428b-b45d-ca8428f561c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:50.681253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e986aa-1a0d-428b-b45d-ca8428f561c9 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:50.681528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e986aa-1a0d-428b-b45d-ca8428f561c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:50.681797 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e986aa-1a0d-428b-b45d-ca8428f561c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:50.692343 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e986aa-1a0d-428b-b45d-ca8428f561c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:50.697089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-d7e986aa-1a0d-428b-b45d-ca8428f561c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:50.697089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:50.697089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:50.697089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:50.697089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:50.697089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:50.697089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:50.697089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:50.697089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:50.697089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:50.697089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:50.697089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:50.697089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:50.697089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:50.697089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:50.697089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:50.698240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:50.698240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:50.698240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:50.698240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:50.698240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:50.698240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:50.698240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:50.698240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:50.698240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:50.698240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:50.698240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:50.698240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:50.698240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:50.698240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:50.698240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:50.698240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:50.699374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:50.699374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:50.699374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:50.699374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:50.699374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:50.699374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:50.699374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:50.699374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:50.699374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:50.699374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:50.699374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:50.699374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:50.699374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 542/1080] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:06:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:50.705341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be6f3f-857d-463e-a5e2-c343d13d63ed None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:50.772979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:50.807037 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be6f3f-857d-463e-a5e2-c343d13d63ed None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:50.842150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67d523-cea3-45b2-848f-d1ed773b07e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:50.842781 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67d523-cea3-45b2-848f-d1ed773b07e0 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:50.843048 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67d523-cea3-45b2-848f-d1ed773b07e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:50.843320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67d523-cea3-45b2-848f-d1ed773b07e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:50.843979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67d523-cea3-45b2-848f-d1ed773b07e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:50.844311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67d523-cea3-45b2-848f-d1ed773b07e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:50.845696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67d523-cea3-45b2-848f-d1ed773b07e0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:50.854046 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 539/1081] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:06:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:50.859705 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e986aa-1a0d-428b-b45d-ca8428f561c9 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:50.961092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e986aa-1a0d-428b-b45d-ca8428f561c9 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:50.997777 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37818edc-ba13-4c1f-a886-e45139657093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:50.998411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37818edc-ba13-4c1f-a886-e45139657093 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:50.998692 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37818edc-ba13-4c1f-a886-e45139657093 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:50.999010 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37818edc-ba13-4c1f-a886-e45139657093 None admin] PATH_INFO: `/v3/users/73f5fcea28044a128b957bd075db3de0` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:51.000140 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37818edc-ba13-4c1f-a886-e45139657093 None admin] RBAC: Authorizing `identity:get_user(user_id=73f5fcea28044a128b957bd075db3de0)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:51.002115 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37818edc-ba13-4c1f-a886-e45139657093 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:51.003779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 543/1082] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:06:50 2022] GET /identity/v3/users/73f5fcea28044a128b957bd075db3de0 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:51.009478 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67d523-cea3-45b2-848f-d1ed773b07e0 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:51.109602 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67d523-cea3-45b2-848f-d1ed773b07e0 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:51.138508 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:51.145139 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797f4b8-39f1-4f1b-bf99-8adb9a0e3a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:51.145776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797f4b8-39f1-4f1b-bf99-8adb9a0e3a69 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:51.146061 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797f4b8-39f1-4f1b-bf99-8adb9a0e3a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:51.146336 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797f4b8-39f1-4f1b-bf99-8adb9a0e3a69 None admin] PATH_INFO: `/v3/projects/dc014f087b5944b8843f1932af169153` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:51.147481 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797f4b8-39f1-4f1b-bf99-8adb9a0e3a69 None admin] RBAC: Authorizing `identity:get_project(project_id=dc014f087b5944b8843f1932af169153)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:51.154441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797f4b8-39f1-4f1b-bf99-8adb9a0e3a69 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:51.154441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 540/1083] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:06:51 2022] GET /identity/v3/projects/dc014f087b5944b8843f1932af169153 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:51.156584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37818edc-ba13-4c1f-a886-e45139657093 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:51.182782 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:51.259220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37818edc-ba13-4c1f-a886-e45139657093 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:51.296009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d3965-cbe4-4778-a127-baa1c1036084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:51.296678 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d3965-cbe4-4778-a127-baa1c1036084 None admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:51.296962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d3965-cbe4-4778-a127-baa1c1036084 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:51.297983 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d3965-cbe4-4778-a127-baa1c1036084 None admin] PATH_INFO: `/v3/projects/dc014f087b5944b8843f1932af169153/users/73f5fcea28044a128b957bd075db3de0/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:51.299949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d3965-cbe4-4778-a127-baa1c1036084 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc014f087b5944b8843f1932af169153, user_id=73f5fcea28044a128b957bd075db3de0, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:51.301382 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d3965-cbe4-4778-a127-baa1c1036084 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:51.368077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 544/1084] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:06:51 2022] PUT /identity/v3/projects/dc014f087b5944b8843f1932af169153/users/73f5fcea28044a128b957bd075db3de0/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:06:52.437044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6b09a-6a46-411f-b9ee-cf6c5e8b0e3f None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:52.437576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6b09a-6a46-411f-b9ee-cf6c5e8b0e3f None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:52.437576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6b09a-6a46-411f-b9ee-cf6c5e8b0e3f None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:52.439862 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 541/1085] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:52.444794 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462a91c-8133-4ead-a8a2-9707aab73123 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:52.445256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462a91c-8133-4ead-a8a2-9707aab73123 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:52.445388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462a91c-8133-4ead-a8a2-9707aab73123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:52.463961 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-6462a91c-8133-4ead-a8a2-9707aab73123 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:52.465149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6462a91c-8133-4ead-a8a2-9707aab73123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:52.533451 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 545/1086] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:52.803750 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6b09a-6a46-411f-b9ee-cf6c5e8b0e3f None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:52.909374 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6b09a-6a46-411f-b9ee-cf6c5e8b0e3f None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:52.941698 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1dea51-5ac5-4126-a427-b6bef4872976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:52.942321 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1dea51-5ac5-4126-a427-b6bef4872976 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:52.942595 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1dea51-5ac5-4126-a427-b6bef4872976 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:52.942864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1dea51-5ac5-4126-a427-b6bef4872976 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:52.953227 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1dea51-5ac5-4126-a427-b6bef4872976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:52.958522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-bc1dea51-5ac5-4126-a427-b6bef4872976 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:06:52.958522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:52.958522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:52.958522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:52.958522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:52.958522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:52.958522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:52.958522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:52.958522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:52.958522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:52.958522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:52.958522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:52.958522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:52.958522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:52.958522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:52.958522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:52.959660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:52.959660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:52.959660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:52.959660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:52.959660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:52.959660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:52.959660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:52.959660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:52.959660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:52.959660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:52.959660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:52.959660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:52.959660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:52.959660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:52.959660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:52.959660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:52.960528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:52.960528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:52.960528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:52.960528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:52.960528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:52.960528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:52.960528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:52.960528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:52.960528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:52.960528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:52.960528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:06:52.960528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:52.960528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 542/1087] 10.222.0.59 () {56 vars in 1120 bytes} [Wed Aug 31 19:06:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:52.965793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462a91c-8133-4ead-a8a2-9707aab73123 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:52.988854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:53.080745 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462a91c-8133-4ead-a8a2-9707aab73123 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:53.114770 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1aff5-3bf0-4adb-8c50-5370b3c71d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:53.115401 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1aff5-3bf0-4adb-8c50-5370b3c71d5c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:53.115677 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1aff5-3bf0-4adb-8c50-5370b3c71d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:53.115942 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1aff5-3bf0-4adb-8c50-5370b3c71d5c None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:53.116593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf1aff5-3bf0-4adb-8c50-5370b3c71d5c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:53.117009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf1aff5-3bf0-4adb-8c50-5370b3c71d5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:53.118361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf1aff5-3bf0-4adb-8c50-5370b3c71d5c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:53.131695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 546/1088] 10.222.0.59 () {56 vars in 1115 bytes} [Wed Aug 31 19:06:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:53.132862 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1dea51-5ac5-4126-a427-b6bef4872976 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:53.192053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:53.234280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1dea51-5ac5-4126-a427-b6bef4872976 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:53.256745 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0477761-c38a-4ceb-8d88-f101028e1a1a None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:53.257032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0477761-c38a-4ceb-8d88-f101028e1a1a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:53.257313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0477761-c38a-4ceb-8d88-f101028e1a1a None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:53.258657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 547/1089] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:53.263597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa66cb-c2e5-43ab-8189-56d3e43a8f82 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:53.263888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa66cb-c2e5-43ab-8189-56d3e43a8f82 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:53.264166 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa66cb-c2e5-43ab-8189-56d3e43a8f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:53.269016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c8d24-4288-45ec-b626-7ca197622730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:53.269662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c8d24-4288-45ec-b626-7ca197622730 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:53.269944 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c8d24-4288-45ec-b626-7ca197622730 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:53.270222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c8d24-4288-45ec-b626-7ca197622730 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:53.283706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-f3fa66cb-c2e5-43ab-8189-56d3e43a8f82 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:53.283958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c8d24-4288-45ec-b626-7ca197622730 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:53.284916 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f3fa66cb-c2e5-43ab-8189-56d3e43a8f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:53.285482 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c8d24-4288-45ec-b626-7ca197622730 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:53.301197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-db6c8d24-4288-45ec-b626-7ca197622730 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:53.301197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:53.301197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:53.301197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:53.301197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:53.301197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:53.301197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:53.301197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:53.301197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:53.301197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:53.301197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:53.301197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:53.301197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:53.301197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:53.301197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:53.301197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:53.310351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:53.310351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:53.310351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:53.310351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:53.310351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:53.310351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:53.310351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:53.310351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:53.310351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:53.310351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:53.310351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:53.310351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:53.310351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:53.310351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:53.310351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:53.310351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:53.310351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:53.311315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:53.311315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:53.311315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:53.311315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 543/1090] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:53.311315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c8d24-4288-45ec-b626-7ca197622730 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:53.358998 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 548/1091] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:06:53.411698 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c8d24-4288-45ec-b626-7ca197622730 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:53.422038 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:53.446108 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f3e32-1fb5-40e8-b037-7c155a30d3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:53.446927 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f3e32-1fb5-40e8-b037-7c155a30d3dc None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:53.447267 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f3e32-1fb5-40e8-b037-7c155a30d3dc None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:53.447679 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f3e32-1fb5-40e8-b037-7c155a30d3dc None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:53.448578 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f3e32-1fb5-40e8-b037-7c155a30d3dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:53.448890 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f3e32-1fb5-40e8-b037-7c155a30d3dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:53.450248 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f3e32-1fb5-40e8-b037-7c155a30d3dc None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:53.463821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 544/1092] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:53.470261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa66cb-c2e5-43ab-8189-56d3e43a8f82 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:53.473789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:53.569072 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa66cb-c2e5-43ab-8189-56d3e43a8f82 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:53.603354 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35946dd8-6e68-482b-871f-76af400313d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:53.603981 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35946dd8-6e68-482b-871f-76af400313d2 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:53.604264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35946dd8-6e68-482b-871f-76af400313d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:53.604538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35946dd8-6e68-482b-871f-76af400313d2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:53.612043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35946dd8-6e68-482b-871f-76af400313d2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:53.613432 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35946dd8-6e68-482b-871f-76af400313d2 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:53.625657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-35946dd8-6e68-482b-871f-76af400313d2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:06:53.625657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:53.625657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:53.625657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:53.625657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:53.625657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:53.625657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:53.625657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:53.625657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:53.625657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:53.625657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:53.625657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:53.625657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:06:53.625657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:06:53.625657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:06:53.625657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:53.626691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:53.627685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:53.627685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:53.627685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:06:53.627685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:06:53.627685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:06:53.627685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:06:53.627685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:06:53.627685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:06:53.627685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:06:53.627685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:53.627685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 549/1093] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 31 19:06:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:53.628322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f3e32-1fb5-40e8-b037-7c155a30d3dc None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:53.632731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35946dd8-6e68-482b-871f-76af400313d2 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:53.695692 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:53.737201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35946dd8-6e68-482b-871f-76af400313d2 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:53.742080 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f3e32-1fb5-40e8-b037-7c155a30d3dc None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:53.772330 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebb94f-cca5-4922-bd33-944bba253082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:53.772938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebb94f-cca5-4922-bd33-944bba253082 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:53.773214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebb94f-cca5-4922-bd33-944bba253082 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:53.773483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebb94f-cca5-4922-bd33-944bba253082 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:53.774206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebb94f-cca5-4922-bd33-944bba253082 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:53.774508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebb94f-cca5-4922-bd33-944bba253082 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:53.775955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebb94f-cca5-4922-bd33-944bba253082 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:53.778642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21de77-fb2d-4914-b0a7-60f0ae94c52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:53.779335 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21de77-fb2d-4914-b0a7-60f0ae94c52a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:53.779611 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21de77-fb2d-4914-b0a7-60f0ae94c52a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:53.779934 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21de77-fb2d-4914-b0a7-60f0ae94c52a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:53.791090 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21de77-fb2d-4914-b0a7-60f0ae94c52a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:53.795928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-da21de77-fb2d-4914-b0a7-60f0ae94c52a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:53.795928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:53.795928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:53.795928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:53.795928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:53.795928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:53.795928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:53.795928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:53.795928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:53.795928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:53.795928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:53.795928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:53.795928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:53.795928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:53.795928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:53.795928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:53.797261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:53.797261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:53.797261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:53.797261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:53.797261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:53.797261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:53.797261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:53.797261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:53.797261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:53.797261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:53.797261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:53.797261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:53.797261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:53.797261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:53.797261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:53.797261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:53.798174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:53.798174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:53.798174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:53.798174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:53.798174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:53.798174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:53.798174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:53.798174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:53.798174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:53.798174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:53.798174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:53.798174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:06:53.798174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 545/1094] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:06:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:53.804498 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21de77-fb2d-4914-b0a7-60f0ae94c52a None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:53.804627 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 550/1095] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 31 19:06:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:53.810527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebb94f-cca5-4922-bd33-944bba253082 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:53.846159 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:53.913839 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebb94f-cca5-4922-bd33-944bba253082 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:53.917424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21de77-fb2d-4914-b0a7-60f0ae94c52a None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:53.936360 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:53.948133 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f41887-141b-4b4e-bb2f-496c5261c6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:53.948790 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f41887-141b-4b4e-bb2f-496c5261c6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:53.949071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f41887-141b-4b4e-bb2f-496c5261c6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:53.949344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f41887-141b-4b4e-bb2f-496c5261c6d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:53.953332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb8ae1-b888-4301-8aca-f1b867792500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:53.954706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb8ae1-b888-4301-8aca-f1b867792500 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:53.955076 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb8ae1-b888-4301-8aca-f1b867792500 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:53.955355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb8ae1-b888-4301-8aca-f1b867792500 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:53.956018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb8ae1-b888-4301-8aca-f1b867792500 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:53.956324 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb8ae1-b888-4301-8aca-f1b867792500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:53.957686 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb8ae1-b888-4301-8aca-f1b867792500 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:53.966240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 546/1096] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:06:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:53.971859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb8ae1-b888-4301-8aca-f1b867792500 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:54.021372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f41887-141b-4b4e-bb2f-496c5261c6d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:54.023027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f41887-141b-4b4e-bb2f-496c5261c6d9 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:54.068037 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:54.076599 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb8ae1-b888-4301-8aca-f1b867792500 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:54.111698 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7e5a1-7d37-4ef4-b265-56de3188003a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:54.112334 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7e5a1-7d37-4ef4-b265-56de3188003a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:54.112606 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7e5a1-7d37-4ef4-b265-56de3188003a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:54.112882 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7e5a1-7d37-4ef4-b265-56de3188003a None admin] PATH_INFO: `/v3/users/73f5fcea28044a128b957bd075db3de0` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:54.114000 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7e5a1-7d37-4ef4-b265-56de3188003a None admin] RBAC: Authorizing `identity:get_user(user_id=73f5fcea28044a128b957bd075db3de0)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:54.115453 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7e5a1-7d37-4ef4-b265-56de3188003a None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:54.117058 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 547/1097] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:06:53 2022] GET /identity/v3/users/73f5fcea28044a128b957bd075db3de0 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:54.123158 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7e5a1-7d37-4ef4-b265-56de3188003a None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:54.134469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 551/1098] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:06:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:06:54.135540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f41887-141b-4b4e-bb2f-496c5261c6d9 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:54.159919 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:54.224432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7e5a1-7d37-4ef4-b265-56de3188003a None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:54.236667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f41887-141b-4b4e-bb2f-496c5261c6d9 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:54.259571 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ea166-0926-477b-a8e4-c63f398d225f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:54.260180 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ea166-0926-477b-a8e4-c63f398d225f None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:54.260460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ea166-0926-477b-a8e4-c63f398d225f None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:54.260734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ea166-0926-477b-a8e4-c63f398d225f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:54.273787 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fe17d-2b29-44f2-9de4-9acbcdf4eb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:54.274568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fe17d-2b29-44f2-9de4-9acbcdf4eb7a None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:54.281991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fe17d-2b29-44f2-9de4-9acbcdf4eb7a None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:54.281991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fe17d-2b29-44f2-9de4-9acbcdf4eb7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:54.286721 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fe17d-2b29-44f2-9de4-9acbcdf4eb7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:54.288257 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fe17d-2b29-44f2-9de4-9acbcdf4eb7a None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:54.291853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:54.304556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-d77fe17d-2b29-44f2-9de4-9acbcdf4eb7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:54.304556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:54.304556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:54.304556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:54.304556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:54.304556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:54.304556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:54.304556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:54.304556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:54.304556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:54.304556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:54.304556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:54.304556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:54.304556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:54.304556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:54.304556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:54.306546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:54.306546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:54.306546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:54.306546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:54.306546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:54.306546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:54.306546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:54.306546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:54.306546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:54.306546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:54.306546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:54.306546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:54.306546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:54.306546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:54.306546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:54.306546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:54.306546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:06:54.308456 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:54.308456 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:06:54.308456 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:54.310860 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 552/1099] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:54.311651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fe17d-2b29-44f2-9de4-9acbcdf4eb7a None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:54.346008 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ea166-0926-477b-a8e4-c63f398d225f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:54.347481 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ea166-0926-477b-a8e4-c63f398d225f None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:54.390546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:54.415209 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fe17d-2b29-44f2-9de4-9acbcdf4eb7a None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:54.450132 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 548/1100] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:06:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 19:06:54.450752 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959290e1-f439-4a7c-b2eb-aa2b468db066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:54.451464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959290e1-f439-4a7c-b2eb-aa2b468db066 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:54.451755 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959290e1-f439-4a7c-b2eb-aa2b468db066 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:54.452032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959290e1-f439-4a7c-b2eb-aa2b468db066 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:54.452688 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959290e1-f439-4a7c-b2eb-aa2b468db066 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:54.452998 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959290e1-f439-4a7c-b2eb-aa2b468db066 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:54.454766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959290e1-f439-4a7c-b2eb-aa2b468db066 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:54.456411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ea166-0926-477b-a8e4-c63f398d225f None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:54.468440 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 553/1101] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:54.473717 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959290e1-f439-4a7c-b2eb-aa2b468db066 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:54.517369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:54.557769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ea166-0926-477b-a8e4-c63f398d225f None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:54.574190 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959290e1-f439-4a7c-b2eb-aa2b468db066 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:54.592860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8449b-a604-4e81-85f6-59bdce4015d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:54.593489 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8449b-a604-4e81-85f6-59bdce4015d3 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:54.593773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8449b-a604-4e81-85f6-59bdce4015d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:54.594672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8449b-a604-4e81-85f6-59bdce4015d3 None admin] PATH_INFO: `/v3/projects/dc014f087b5944b8843f1932af169153` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:54.595740 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8449b-a604-4e81-85f6-59bdce4015d3 None admin] RBAC: Authorizing `identity:get_project(project_id=dc014f087b5944b8843f1932af169153)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:54.597162 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8449b-a604-4e81-85f6-59bdce4015d3 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:54.598790 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 549/1102] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:06:54 2022] GET /identity/v3/projects/dc014f087b5944b8843f1932af169153 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:54.604753 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8449b-a604-4e81-85f6-59bdce4015d3 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:54.607893 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb56e9-4ee1-417a-bc59-70d531cbb1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:54.608523 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb56e9-4ee1-417a-bc59-70d531cbb1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:54.608797 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb56e9-4ee1-417a-bc59-70d531cbb1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:54.609068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb56e9-4ee1-417a-bc59-70d531cbb1b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:54.612105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:54.617459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb56e9-4ee1-417a-bc59-70d531cbb1b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:54.618917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb56e9-4ee1-417a-bc59-70d531cbb1b9 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:54.630842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-9bfb56e9-4ee1-417a-bc59-70d531cbb1b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:54.630842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:54.630842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:54.630842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:54.630842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:54.630842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:54.630842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:54.630842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:54.630842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:54.630842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:54.630842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:54.630842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:54.630842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:06:54.630842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:06:54.630842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:06:54.632018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:06:54.632018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:54.632018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:54.632018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:54.632018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:54.632018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:54.632018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:54.632018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:54.632018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:54.632018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:54.632018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:54.632018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:54.632018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:54.632018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:54.632018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:54.632018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:54.632018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:54.633165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:06:54.633165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:06:54.633165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:06:54.633165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:06:54.633165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:06:54.633165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:06:54.633165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:06:54.633165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:06:54.637079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 554/1103] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:06:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:06:54.637855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb56e9-4ee1-417a-bc59-70d531cbb1b9 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:54.713082 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8449b-a604-4e81-85f6-59bdce4015d3 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:54.739855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb56e9-4ee1-417a-bc59-70d531cbb1b9 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:54.742290 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:54.750810 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f009a51-6039-4925-9c8b-e597d7edd8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:54.754333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f009a51-6039-4925-9c8b-e597d7edd8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:54.754333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f009a51-6039-4925-9c8b-e597d7edd8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:54.754333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f009a51-6039-4925-9c8b-e597d7edd8b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:54.754333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f009a51-6039-4925-9c8b-e597d7edd8b2 None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=dc014f087b5944b8843f1932af169153, user.id=73f5fcea28044a128b957bd075db3de0) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:54.754333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f009a51-6039-4925-9c8b-e597d7edd8b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:54.754333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f009a51-6039-4925-9c8b-e597d7edd8b2 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:54.759889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 550/1104] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:06:54 2022] GET /identity/v3/role_assignments?user.id=73f5fcea28044a128b957bd075db3de0&scope.project.id=dc014f087b5944b8843f1932af169153&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:54.775252 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691ae4e-4258-4df0-be9c-6e5eed332e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:54.775809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691ae4e-4258-4df0-be9c-6e5eed332e3c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:54.776092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691ae4e-4258-4df0-be9c-6e5eed332e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:54.776370 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691ae4e-4258-4df0-be9c-6e5eed332e3c None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:54.777095 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691ae4e-4258-4df0-be9c-6e5eed332e3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:54.777464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691ae4e-4258-4df0-be9c-6e5eed332e3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:54.779039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691ae4e-4258-4df0-be9c-6e5eed332e3c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:54.793933 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 555/1105] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:06:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:06:54.801333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f009a51-6039-4925-9c8b-e597d7edd8b2 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:54.910997 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f009a51-6039-4925-9c8b-e597d7edd8b2 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:54.946740 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17afa0-1fd9-4488-a4cc-efd723235050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:54.947393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17afa0-1fd9-4488-a4cc-efd723235050 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:54.947675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17afa0-1fd9-4488-a4cc-efd723235050 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:54.947941 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17afa0-1fd9-4488-a4cc-efd723235050 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:54.948659 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17afa0-1fd9-4488-a4cc-efd723235050 None admin] RBAC: Adding query filter params (role.id=e22b60b1c78d45b29c81bcce9064de49, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=d461422ce98442bcbe6b254a8dad5e8a) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:54.948964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17afa0-1fd9-4488-a4cc-efd723235050 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:54.950372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17afa0-1fd9-4488-a4cc-efd723235050 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:54.957055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 551/1106] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:06:54 2022] GET /identity/v3/role_assignments?user.id=d461422ce98442bcbe6b254a8dad5e8a&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=e22b60b1c78d45b29c81bcce9064de49 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:56.689762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b55919-52ad-45bb-ac47-8d77031b6d42 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:56.690302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b55919-52ad-45bb-ac47-8d77031b6d42 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:56.690392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b55919-52ad-45bb-ac47-8d77031b6d42 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:56.691790 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 556/1107] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:56.696967 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690d6d1-902b-4573-bbdd-b9150077aaba None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:56.697401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690d6d1-902b-4573-bbdd-b9150077aaba None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:56.697527 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690d6d1-902b-4573-bbdd-b9150077aaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:56.717015 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-d690d6d1-902b-4573-bbdd-b9150077aaba None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:56.718210 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d690d6d1-902b-4573-bbdd-b9150077aaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:56.772005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:56.789840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 552/1108] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:57.064611 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b55919-52ad-45bb-ac47-8d77031b6d42 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:57.122828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:57.168066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b55919-52ad-45bb-ac47-8d77031b6d42 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:57.201178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b977d-d3bc-401e-baf4-f8962a94a035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:57.201801 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b977d-d3bc-401e-baf4-f8962a94a035 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:57.202079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b977d-d3bc-401e-baf4-f8962a94a035 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:57.202349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b977d-d3bc-401e-baf4-f8962a94a035 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:57.204011 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b977d-d3bc-401e-baf4-f8962a94a035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:57.205439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b977d-d3bc-401e-baf4-f8962a94a035 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:57.207035 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 557/1109] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:06:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:57.212830 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690d6d1-902b-4573-bbdd-b9150077aaba None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:57.330117 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690d6d1-902b-4573-bbdd-b9150077aaba None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:57.374083 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53a836-fb94-476d-bd1c-f300c49643f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:57.376409 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53a836-fb94-476d-bd1c-f300c49643f3 None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:57.376824 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53a836-fb94-476d-bd1c-f300c49643f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:57.377122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53a836-fb94-476d-bd1c-f300c49643f3 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:57.377798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53a836-fb94-476d-bd1c-f300c49643f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:57.379909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53a836-fb94-476d-bd1c-f300c49643f3 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:57.499506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 553/1110] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:06:57 2022] POST /identity/v3/users => generated 286 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:59.631957 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be061ad-13b9-47a7-9732-a78fa75fac2c None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:59.632496 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be061ad-13b9-47a7-9732-a78fa75fac2c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:59.632496 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be061ad-13b9-47a7-9732-a78fa75fac2c None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:59.633825 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 558/1111] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:59.639077 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afbcf9-5732-4547-b2d7-a3814f9b26ae None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:59.639505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afbcf9-5732-4547-b2d7-a3814f9b26ae None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:59.639630 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afbcf9-5732-4547-b2d7-a3814f9b26ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:59.657723 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-e9afbcf9-5732-4547-b2d7-a3814f9b26ae None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:59.658950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e9afbcf9-5732-4547-b2d7-a3814f9b26ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:59.694658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:59.726050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 554/1112] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:59.999998 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be061ad-13b9-47a7-9732-a78fa75fac2c None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:00.098838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be061ad-13b9-47a7-9732-a78fa75fac2c None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:00.121792 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:00.133394 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa8237-75b5-4d23-a6fd-3e5ce55bd31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:00.137857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa8237-75b5-4d23-a6fd-3e5ce55bd31b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:00.137857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa8237-75b5-4d23-a6fd-3e5ce55bd31b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:00.137857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa8237-75b5-4d23-a6fd-3e5ce55bd31b None admin] PATH_INFO: `/v3/roles/e14f37f4a3a24f8da0f757aabb1f39be` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:00.137857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa8237-75b5-4d23-a6fd-3e5ce55bd31b None admin] RBAC: Authorizing `identity:get_role(role_id=e14f37f4a3a24f8da0f757aabb1f39be)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:00.137857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa8237-75b5-4d23-a6fd-3e5ce55bd31b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:00.138584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 559/1113] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:06:59 2022] GET /identity/v3/roles/e14f37f4a3a24f8da0f757aabb1f39be => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:00.144990 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afbcf9-5732-4547-b2d7-a3814f9b26ae None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:00.243025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afbcf9-5732-4547-b2d7-a3814f9b26ae None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:00.276822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867b953-2d79-4bc2-8298-e1ca12211977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:00.277467 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867b953-2d79-4bc2-8298-e1ca12211977 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:00.277745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867b953-2d79-4bc2-8298-e1ca12211977 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:00.278014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867b953-2d79-4bc2-8298-e1ca12211977 None admin] PATH_INFO: `/v3/users/2c54fb4bac204c219fa802efa08f1070` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:00.292434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d867b953-2d79-4bc2-8298-e1ca12211977 None admin] RBAC: Authorizing `identity:get_user(user_id=2c54fb4bac204c219fa802efa08f1070)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:00.293879 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d867b953-2d79-4bc2-8298-e1ca12211977 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:00.295653 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 555/1114] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:07:00 2022] GET /identity/v3/users/2c54fb4bac204c219fa802efa08f1070 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:00.301859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa8237-75b5-4d23-a6fd-3e5ce55bd31b None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:00.402751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa8237-75b5-4d23-a6fd-3e5ce55bd31b None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:00.436734 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cefbfa-b367-4b2a-a6d3-bf9dce754a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:00.437340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cefbfa-b367-4b2a-a6d3-bf9dce754a71 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:00.437670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cefbfa-b367-4b2a-a6d3-bf9dce754a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:00.438030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cefbfa-b367-4b2a-a6d3-bf9dce754a71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:00.494787 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:00.504902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cefbfa-b367-4b2a-a6d3-bf9dce754a71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:00.506369 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cefbfa-b367-4b2a-a6d3-bf9dce754a71 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:00.603533 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 560/1115] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:07:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 19:07:00.609571 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867b953-2d79-4bc2-8298-e1ca12211977 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:00.640121 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:00.708890 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867b953-2d79-4bc2-8298-e1ca12211977 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:00.741746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ef040-e3e5-43b8-a79c-d5f5561bc23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:00.742369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ef040-e3e5-43b8-a79c-d5f5561bc23a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:00.742649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ef040-e3e5-43b8-a79c-d5f5561bc23a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:00.742949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ef040-e3e5-43b8-a79c-d5f5561bc23a None admin] PATH_INFO: `/v3/projects/dc014f087b5944b8843f1932af169153` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:00.744054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ef040-e3e5-43b8-a79c-d5f5561bc23a None admin] RBAC: Authorizing `identity:get_project(project_id=dc014f087b5944b8843f1932af169153)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:00.745482 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ef040-e3e5-43b8-a79c-d5f5561bc23a None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:00.747164 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 556/1116] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:07:00 2022] GET /identity/v3/projects/dc014f087b5944b8843f1932af169153 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:00.753986 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cefbfa-b367-4b2a-a6d3-bf9dce754a71 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:00.846711 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:00.860836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cefbfa-b367-4b2a-a6d3-bf9dce754a71 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:00.891824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4af644-228a-4adb-ba68-98cb7b79eb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:00.892427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4af644-228a-4adb-ba68-98cb7b79eb93 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:00.892708 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4af644-228a-4adb-ba68-98cb7b79eb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:00.892979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4af644-228a-4adb-ba68-98cb7b79eb93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:00.893702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4af644-228a-4adb-ba68-98cb7b79eb93 None admin] RBAC: Adding query filter params (role.id=e14f37f4a3a24f8da0f757aabb1f39be, scope.project.id=dc014f087b5944b8843f1932af169153, user.id=2c54fb4bac204c219fa802efa08f1070) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:00.894011 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4af644-228a-4adb-ba68-98cb7b79eb93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:00.895485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4af644-228a-4adb-ba68-98cb7b79eb93 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:00.900717 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 561/1117] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:07:00 2022] GET /identity/v3/role_assignments?user.id=2c54fb4bac204c219fa802efa08f1070&scope.project.id=dc014f087b5944b8843f1932af169153&role.id=e14f37f4a3a24f8da0f757aabb1f39be => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:02.776425 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f56ba-07e7-44d3-b6d9-ce117dcb6b4e None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:02.776969 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f56ba-07e7-44d3-b6d9-ce117dcb6b4e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:02.776969 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f56ba-07e7-44d3-b6d9-ce117dcb6b4e None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:02.778320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 557/1118] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:02.784213 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea684f3d-0afd-432e-b585-dbb48076bebf None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:02.784668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea684f3d-0afd-432e-b585-dbb48076bebf None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:02.784790 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea684f3d-0afd-432e-b585-dbb48076bebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:02.803604 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-ea684f3d-0afd-432e-b585-dbb48076bebf None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:02.804634 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ea684f3d-0afd-432e-b585-dbb48076bebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:02.871216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 562/1119] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:03.141894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f56ba-07e7-44d3-b6d9-ce117dcb6b4e None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:03.239133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:03.243811 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f56ba-07e7-44d3-b6d9-ce117dcb6b4e None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:03.277024 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590e0ca-2936-44de-9439-be65efe0158a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:03.277657 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590e0ca-2936-44de-9439-be65efe0158a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:03.277938 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590e0ca-2936-44de-9439-be65efe0158a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:03.278211 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590e0ca-2936-44de-9439-be65efe0158a None admin] PATH_INFO: `/v3/roles/e14f37f4a3a24f8da0f757aabb1f39be` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:03.279720 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590e0ca-2936-44de-9439-be65efe0158a None admin] RBAC: Authorizing `identity:get_role(role_id=e14f37f4a3a24f8da0f757aabb1f39be)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:03.281125 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590e0ca-2936-44de-9439-be65efe0158a None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:03.283332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 558/1120] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:07:03 2022] GET /identity/v3/roles/e14f37f4a3a24f8da0f757aabb1f39be => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:03.288992 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea684f3d-0afd-432e-b585-dbb48076bebf None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:03.371757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:03.387265 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea684f3d-0afd-432e-b585-dbb48076bebf None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:03.422290 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1a8e4-ef5e-497d-bbc2-f5374c28fad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:03.422832 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1a8e4-ef5e-497d-bbc2-f5374c28fad3 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:03.423144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1a8e4-ef5e-497d-bbc2-f5374c28fad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:03.423424 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1a8e4-ef5e-497d-bbc2-f5374c28fad3 None admin] PATH_INFO: `/v3/users/2c54fb4bac204c219fa802efa08f1070` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:03.424548 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1a8e4-ef5e-497d-bbc2-f5374c28fad3 None admin] RBAC: Authorizing `identity:get_user(user_id=2c54fb4bac204c219fa802efa08f1070)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:03.426032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1a8e4-ef5e-497d-bbc2-f5374c28fad3 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:03.427649 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 563/1121] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:07:03 2022] GET /identity/v3/users/2c54fb4bac204c219fa802efa08f1070 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:03.433658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590e0ca-2936-44de-9439-be65efe0158a None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:03.532361 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590e0ca-2936-44de-9439-be65efe0158a None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:03.565762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503f2c2-0965-4ab8-9781-57984c6a0fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:03.566385 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503f2c2-0965-4ab8-9781-57984c6a0fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:03.566664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503f2c2-0965-4ab8-9781-57984c6a0fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:03.566967 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503f2c2-0965-4ab8-9781-57984c6a0fa4 None admin] PATH_INFO: `/v3/projects/dc014f087b5944b8843f1932af169153` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:03.568066 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503f2c2-0965-4ab8-9781-57984c6a0fa4 None admin] RBAC: Authorizing `identity:get_project(project_id=dc014f087b5944b8843f1932af169153)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:03.569476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503f2c2-0965-4ab8-9781-57984c6a0fa4 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:03.571118 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 559/1122] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:07:03 2022] GET /identity/v3/projects/dc014f087b5944b8843f1932af169153 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:03.577167 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1a8e4-ef5e-497d-bbc2-f5374c28fad3 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:03.676450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1a8e4-ef5e-497d-bbc2-f5374c28fad3 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:03.711050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345abb63-2727-4936-b840-43a37a530dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:03.711703 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345abb63-2727-4936-b840-43a37a530dfc None admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:03.711986 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345abb63-2727-4936-b840-43a37a530dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:03.712263 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345abb63-2727-4936-b840-43a37a530dfc None admin] PATH_INFO: `/v3/projects/dc014f087b5944b8843f1932af169153/users/2c54fb4bac204c219fa802efa08f1070/roles/e14f37f4a3a24f8da0f757aabb1f39be` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:03.714395 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345abb63-2727-4936-b840-43a37a530dfc None admin] RBAC: Authorizing `identity:create_grant(project_id=dc014f087b5944b8843f1932af169153, user_id=2c54fb4bac204c219fa802efa08f1070, role_id=e14f37f4a3a24f8da0f757aabb1f39be)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:03.715854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345abb63-2727-4936-b840-43a37a530dfc None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:03.799506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 564/1123] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:07:03 2022] PUT /identity/v3/projects/dc014f087b5944b8843f1932af169153/users/2c54fb4bac204c219fa802efa08f1070/roles/e14f37f4a3a24f8da0f757aabb1f39be => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:07:05.679958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1208079-0e60-480f-be5b-6b430b860d4d None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:05.680514 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1208079-0e60-480f-be5b-6b430b860d4d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:05.680514 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1208079-0e60-480f-be5b-6b430b860d4d None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:05.681855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 560/1124] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:05.687162 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f22cd-563f-4cb4-8985-83f5e70d0fc1 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:05.687610 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f22cd-563f-4cb4-8985-83f5e70d0fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:05.687948 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f22cd-563f-4cb4-8985-83f5e70d0fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:05.706227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-3a2f22cd-563f-4cb4-8985-83f5e70d0fc1 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:05.707457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3a2f22cd-563f-4cb4-8985-83f5e70d0fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:05.714474 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:05.772657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 565/1125] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:06.037137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1208079-0e60-480f-be5b-6b430b860d4d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:06.073285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:06.139882 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1208079-0e60-480f-be5b-6b430b860d4d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:06.174077 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c4cf7-5475-42ff-a132-d5b107b70d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:06.174699 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c4cf7-5475-42ff-a132-d5b107b70d7b None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:06.175010 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c4cf7-5475-42ff-a132-d5b107b70d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:06.175288 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c4cf7-5475-42ff-a132-d5b107b70d7b None admin] PATH_INFO: `/v3/roles/e14f37f4a3a24f8da0f757aabb1f39be` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:06.176678 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c4cf7-5475-42ff-a132-d5b107b70d7b None admin] RBAC: Authorizing `identity:get_role(role_id=e14f37f4a3a24f8da0f757aabb1f39be)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:06.178039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c4cf7-5475-42ff-a132-d5b107b70d7b None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:06.179401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 561/1126] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:07:06 2022] GET /identity/v3/roles/e14f37f4a3a24f8da0f757aabb1f39be => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:06.185233 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f22cd-563f-4cb4-8985-83f5e70d0fc1 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:06.284600 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f22cd-563f-4cb4-8985-83f5e70d0fc1 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:06.323307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892d596-fcf0-4f77-a286-2e99e066a83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:06.323938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892d596-fcf0-4f77-a286-2e99e066a83c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:06.324212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892d596-fcf0-4f77-a286-2e99e066a83c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:06.324481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892d596-fcf0-4f77-a286-2e99e066a83c None admin] PATH_INFO: `/v3/users/2c54fb4bac204c219fa802efa08f1070` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:06.325676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892d596-fcf0-4f77-a286-2e99e066a83c None admin] RBAC: Authorizing `identity:get_user(user_id=2c54fb4bac204c219fa802efa08f1070)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:06.327118 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892d596-fcf0-4f77-a286-2e99e066a83c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:06.328851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 566/1127] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:07:06 2022] GET /identity/v3/users/2c54fb4bac204c219fa802efa08f1070 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:06.334681 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c4cf7-5475-42ff-a132-d5b107b70d7b None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:06.435327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c4cf7-5475-42ff-a132-d5b107b70d7b None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:06.436852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:06.471107 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6432f42-aa84-4521-b7c9-53612c9d0acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:06.471713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6432f42-aa84-4521-b7c9-53612c9d0acc None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:06.471996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6432f42-aa84-4521-b7c9-53612c9d0acc None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:06.472271 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6432f42-aa84-4521-b7c9-53612c9d0acc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:06.539249 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6432f42-aa84-4521-b7c9-53612c9d0acc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:06.540667 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6432f42-aa84-4521-b7c9-53612c9d0acc None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:06.637651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 562/1128] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:07:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:07:06.644384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892d596-fcf0-4f77-a286-2e99e066a83c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:06.656919 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:06.741598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892d596-fcf0-4f77-a286-2e99e066a83c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:06.775995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4e2df-4a4d-4f87-9122-242a48174d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:06.776617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4e2df-4a4d-4f87-9122-242a48174d30 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:06.776901 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4e2df-4a4d-4f87-9122-242a48174d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:06.777176 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4e2df-4a4d-4f87-9122-242a48174d30 None admin] PATH_INFO: `/v3/projects/dc014f087b5944b8843f1932af169153` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:06.778293 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4e2df-4a4d-4f87-9122-242a48174d30 None admin] RBAC: Authorizing `identity:get_project(project_id=dc014f087b5944b8843f1932af169153)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:06.779751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4e2df-4a4d-4f87-9122-242a48174d30 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:06.781650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 567/1129] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:07:06 2022] GET /identity/v3/projects/dc014f087b5944b8843f1932af169153 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:06.787832 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6432f42-aa84-4521-b7c9-53612c9d0acc None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:06.790321 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:06.893251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6432f42-aa84-4521-b7c9-53612c9d0acc None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:06.926785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0345b-8b20-4f5f-94fb-db06efcf7f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:06.927421 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0345b-8b20-4f5f-94fb-db06efcf7f6c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:06.927704 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0345b-8b20-4f5f-94fb-db06efcf7f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:06.927983 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0345b-8b20-4f5f-94fb-db06efcf7f6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:06.928705 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0345b-8b20-4f5f-94fb-db06efcf7f6c None admin] RBAC: Adding query filter params (role.id=e14f37f4a3a24f8da0f757aabb1f39be, scope.project.id=dc014f087b5944b8843f1932af169153, user.id=2c54fb4bac204c219fa802efa08f1070) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:06.929015 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0345b-8b20-4f5f-94fb-db06efcf7f6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:06.930431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0345b-8b20-4f5f-94fb-db06efcf7f6c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:06.936078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 563/1130] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:07:06 2022] GET /identity/v3/role_assignments?user.id=2c54fb4bac204c219fa802efa08f1070&scope.project.id=dc014f087b5944b8843f1932af169153&role.id=e14f37f4a3a24f8da0f757aabb1f39be => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:08.895779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aad100-9d07-4a03-b620-f59b183a3b74 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:08.896349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aad100-9d07-4a03-b620-f59b183a3b74 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:08.896349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aad100-9d07-4a03-b620-f59b183a3b74 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:08.897660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 568/1131] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:08.902828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975b0d9-8308-4ae9-9d06-8c110b25bdc0 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:08.903285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975b0d9-8308-4ae9-9d06-8c110b25bdc0 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:08.903421 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975b0d9-8308-4ae9-9d06-8c110b25bdc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:08.921969 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-7975b0d9-8308-4ae9-9d06-8c110b25bdc0 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:08.923184 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-7975b0d9-8308-4ae9-9d06-8c110b25bdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:08.981286 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:08.989022 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 564/1132] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:09.264537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aad100-9d07-4a03-b620-f59b183a3b74 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:09.345522 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:09.367290 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aad100-9d07-4a03-b620-f59b183a3b74 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:09.400978 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aea4f5-f8b9-4d6b-8422-2adb19e368cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:09.401607 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aea4f5-f8b9-4d6b-8422-2adb19e368cf None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:09.401888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aea4f5-f8b9-4d6b-8422-2adb19e368cf None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:09.402162 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aea4f5-f8b9-4d6b-8422-2adb19e368cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:09.403312 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aea4f5-f8b9-4d6b-8422-2adb19e368cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:09.404745 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aea4f5-f8b9-4d6b-8422-2adb19e368cf None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:09.407125 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 569/1133] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:09.413069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975b0d9-8308-4ae9-9d06-8c110b25bdc0 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:09.510892 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975b0d9-8308-4ae9-9d06-8c110b25bdc0 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:09.545065 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec140e82-eb75-476d-8c31-72cd3013c9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:09.545596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec140e82-eb75-476d-8c31-72cd3013c9ea None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:09.545872 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec140e82-eb75-476d-8c31-72cd3013c9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:09.546140 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec140e82-eb75-476d-8c31-72cd3013c9ea None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:09.546812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec140e82-eb75-476d-8c31-72cd3013c9ea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:09.548382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec140e82-eb75-476d-8c31-72cd3013c9ea None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:09.636223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 565/1134] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 31 19:07:09 2022] POST /identity/v3/projects => generated 311 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:07:11.565351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe57ca-4b3a-458b-a7fe-f56e0a11cb08 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:11.565859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe57ca-4b3a-458b-a7fe-f56e0a11cb08 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:11.565859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe57ca-4b3a-458b-a7fe-f56e0a11cb08 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:11.567180 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 570/1135] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:11.572540 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3009d3a-9bd8-48bd-9232-b1c119c80c3b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:11.572949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3009d3a-9bd8-48bd-9232-b1c119c80c3b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:11.573093 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3009d3a-9bd8-48bd-9232-b1c119c80c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:11.591703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-f3009d3a-9bd8-48bd-9232-b1c119c80c3b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:11.592888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f3009d3a-9bd8-48bd-9232-b1c119c80c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:11.637309 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:11.661332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 566/1136] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:07:11.930700 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe57ca-4b3a-458b-a7fe-f56e0a11cb08 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:12.031880 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe57ca-4b3a-458b-a7fe-f56e0a11cb08 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:12.065299 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd6353-1596-475c-a8e4-aaf51d9e61be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:12.065924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd6353-1596-475c-a8e4-aaf51d9e61be None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:12.066205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd6353-1596-475c-a8e4-aaf51d9e61be None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:12.066477 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd6353-1596-475c-a8e4-aaf51d9e61be None admin] PATH_INFO: `/v3/domains/default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:12.067604 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd6353-1596-475c-a8e4-aaf51d9e61be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:12.069039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd6353-1596-475c-a8e4-aaf51d9e61be None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:12.070602 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 571/1137] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:12.076381 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3009d3a-9bd8-48bd-9232-b1c119c80c3b None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:12.174491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3009d3a-9bd8-48bd-9232-b1c119c80c3b None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:12.208238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e289a2b-6bdd-45e6-878b-b412fdde4a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:12.208775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e289a2b-6bdd-45e6-878b-b412fdde4a1e None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:12.209057 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e289a2b-6bdd-45e6-878b-b412fdde4a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:12.209329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e289a2b-6bdd-45e6-878b-b412fdde4a1e None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:12.209978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e289a2b-6bdd-45e6-878b-b412fdde4a1e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:12.211435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e289a2b-6bdd-45e6-878b-b412fdde4a1e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:12.321569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 567/1138] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:07:12 2022] POST /identity/v3/users => generated 286 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:07:14.284722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07369610-7cb7-4590-b603-2b5cd4ea8eb2 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:14.285269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07369610-7cb7-4590-b603-2b5cd4ea8eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:14.285427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07369610-7cb7-4590-b603-2b5cd4ea8eb2 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:14.286853 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 572/1139] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:14.292242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16756812-201c-4013-b689-5a75f291dc96 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:14.292653 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16756812-201c-4013-b689-5a75f291dc96 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:14.292778 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16756812-201c-4013-b689-5a75f291dc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:14.310513 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-16756812-201c-4013-b689-5a75f291dc96 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:14.311723 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-16756812-201c-4013-b689-5a75f291dc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:14.339034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:14.379086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 568/1140] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:07:14.649634 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07369610-7cb7-4590-b603-2b5cd4ea8eb2 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:14.666019 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:14.752087 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07369610-7cb7-4590-b603-2b5cd4ea8eb2 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:14.785791 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ea98a-03fc-4cc9-b0ae-3cac0a678212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:14.786427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ea98a-03fc-4cc9-b0ae-3cac0a678212 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:14.786710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ea98a-03fc-4cc9-b0ae-3cac0a678212 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:14.787020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ea98a-03fc-4cc9-b0ae-3cac0a678212 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:14.797028 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ea98a-03fc-4cc9-b0ae-3cac0a678212 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:07:14.801876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-a52ea98a-03fc-4cc9-b0ae-3cac0a678212 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:14.801876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:14.801876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:14.801876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:14.801876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:14.801876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:14.801876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:14.801876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:14.801876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:14.801876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:14.801876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:14.801876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:14.801876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:07:14.801876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:07:14.801876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:07:14.801876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:07:14.803096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:07:14.803096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:07:14.803096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:07:14.803096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:07:14.803096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:14.803096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:14.803096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:14.803096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:14.803096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:14.803096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:14.803096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:14.803096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:14.803096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:14.803096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:14.803096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:14.803096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:14.804225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:14.804225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:14.804225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:14.804225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:14.804225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:07:14.804225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:07:14.804225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:07:14.804225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:07:14.804225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:07:14.804225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:07:14.804225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:14.804225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:07:14.804225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 573/1141] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:07:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:14.809006 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16756812-201c-4013-b689-5a75f291dc96 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:14.908744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16756812-201c-4013-b689-5a75f291dc96 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:14.942833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad77972-128c-4c1c-bdfe-58142aa4446e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:14.943479 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad77972-128c-4c1c-bdfe-58142aa4446e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:14.943762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad77972-128c-4c1c-bdfe-58142aa4446e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:14.944038 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad77972-128c-4c1c-bdfe-58142aa4446e None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:14.944709 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad77972-128c-4c1c-bdfe-58142aa4446e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:14.945019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad77972-128c-4c1c-bdfe-58142aa4446e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:14.946385 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad77972-128c-4c1c-bdfe-58142aa4446e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:14.955253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 569/1142] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:07:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:14.961099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ea98a-03fc-4cc9-b0ae-3cac0a678212 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:15.037653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:15.062739 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ea98a-03fc-4cc9-b0ae-3cac0a678212 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:15.097638 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f32afb-ce3f-414b-8831-70c1167324f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:15.098267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f32afb-ce3f-414b-8831-70c1167324f4 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:15.098550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f32afb-ce3f-414b-8831-70c1167324f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:15.098826 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f32afb-ce3f-414b-8831-70c1167324f4 None admin] PATH_INFO: `/v3/users/7e4391c9e4604e029530ed49a6723012` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:15.112212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f32afb-ce3f-414b-8831-70c1167324f4 None admin] RBAC: Authorizing `identity:get_user(user_id=7e4391c9e4604e029530ed49a6723012)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:15.113651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f32afb-ce3f-414b-8831-70c1167324f4 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:15.115338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 574/1143] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:07:14 2022] GET /identity/v3/users/7e4391c9e4604e029530ed49a6723012 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:15.121498 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad77972-128c-4c1c-bdfe-58142aa4446e None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:15.150244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:15.225517 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad77972-128c-4c1c-bdfe-58142aa4446e None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:15.259530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c50924-a859-4536-903b-b2d7e16d9e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:15.260128 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c50924-a859-4536-903b-b2d7e16d9e5c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:15.260405 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c50924-a859-4536-903b-b2d7e16d9e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:15.260701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c50924-a859-4536-903b-b2d7e16d9e5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:15.328357 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c50924-a859-4536-903b-b2d7e16d9e5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:15.329792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c50924-a859-4536-903b-b2d7e16d9e5c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:15.355380 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:15.426224 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 570/1144] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:07:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:07:15.432202 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f32afb-ce3f-414b-8831-70c1167324f4 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:15.530544 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f32afb-ce3f-414b-8831-70c1167324f4 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:15.560789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:15.566364 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc303de-51b7-4870-a9f4-ab42141b32f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:15.567030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc303de-51b7-4870-a9f4-ab42141b32f1 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:15.567330 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc303de-51b7-4870-a9f4-ab42141b32f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:15.567600 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc303de-51b7-4870-a9f4-ab42141b32f1 None admin] PATH_INFO: `/v3/projects/dcdb536cf9ab436cbd0169a8eca319fd` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:15.568710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc303de-51b7-4870-a9f4-ab42141b32f1 None admin] RBAC: Authorizing `identity:get_project(project_id=dcdb536cf9ab436cbd0169a8eca319fd)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:15.570197 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc303de-51b7-4870-a9f4-ab42141b32f1 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:15.572133 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 575/1145] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:07:15 2022] GET /identity/v3/projects/dcdb536cf9ab436cbd0169a8eca319fd => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:15.578396 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c50924-a859-4536-903b-b2d7e16d9e5c None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:15.678083 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c50924-a859-4536-903b-b2d7e16d9e5c None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:15.703467 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:15.712013 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32477a-ae99-44b0-8a50-9f9be9f41672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:15.712615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32477a-ae99-44b0-8a50-9f9be9f41672 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:15.712929 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32477a-ae99-44b0-8a50-9f9be9f41672 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:15.717469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32477a-ae99-44b0-8a50-9f9be9f41672 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:15.717469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32477a-ae99-44b0-8a50-9f9be9f41672 None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=dcdb536cf9ab436cbd0169a8eca319fd, user.id=7e4391c9e4604e029530ed49a6723012) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:15.717469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32477a-ae99-44b0-8a50-9f9be9f41672 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:15.717469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32477a-ae99-44b0-8a50-9f9be9f41672 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:15.721937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 571/1146] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:07:15 2022] GET /identity/v3/role_assignments?user.id=7e4391c9e4604e029530ed49a6723012&scope.project.id=dcdb536cf9ab436cbd0169a8eca319fd&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:17.613711 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1afc4-a2b7-4761-bc95-fed4f147ab60 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:17.614313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1afc4-a2b7-4761-bc95-fed4f147ab60 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:17.614313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1afc4-a2b7-4761-bc95-fed4f147ab60 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:17.615625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 576/1147] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:17.620884 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f00176-d984-4567-91e2-34b3b7efe766 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:17.621306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f00176-d984-4567-91e2-34b3b7efe766 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:17.621408 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f00176-d984-4567-91e2-34b3b7efe766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:17.639341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-a6f00176-d984-4567-91e2-34b3b7efe766 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:17.641049 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a6f00176-d984-4567-91e2-34b3b7efe766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:17.707523 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 572/1148] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:07:17.974206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1afc4-a2b7-4761-bc95-fed4f147ab60 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:18.075743 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1afc4-a2b7-4761-bc95-fed4f147ab60 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:18.109255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430e86f-bcd6-44c2-a1d0-934e5642aeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:18.110136 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430e86f-bcd6-44c2-a1d0-934e5642aeac None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:18.110416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430e86f-bcd6-44c2-a1d0-934e5642aeac None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:18.110697 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430e86f-bcd6-44c2-a1d0-934e5642aeac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:18.120744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430e86f-bcd6-44c2-a1d0-934e5642aeac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:07:18.126576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-e430e86f-bcd6-44c2-a1d0-934e5642aeac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:18.126576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:18.126576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:18.126576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:18.126576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:18.126576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:18.126576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:18.126576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:18.126576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:18.126576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:18.126576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:18.126576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:18.126576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:07:18.126576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:07:18.126576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:07:18.126576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:07:18.127776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:07:18.127776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:07:18.127776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:07:18.127776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:07:18.127776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:18.127776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:18.127776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:18.127776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:18.127776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:18.127776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:18.127776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:18.127776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:18.127776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:18.127776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:18.127776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:18.127776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:18.129058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:18.129058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:18.129058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:18.129058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:18.129058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:07:18.129058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:07:18.129058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:07:18.129058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:07:18.129058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:07:18.129058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:07:18.129058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:18.129058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:07:18.129058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 577/1149] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:07:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:18.133795 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f00176-d984-4567-91e2-34b3b7efe766 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:18.234240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f00176-d984-4567-91e2-34b3b7efe766 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:18.254778 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:18.268749 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f89c1-5212-41c8-88f7-c2ba01412e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:18.272155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f89c1-5212-41c8-88f7-c2ba01412e81 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:18.272155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f89c1-5212-41c8-88f7-c2ba01412e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:18.272155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f89c1-5212-41c8-88f7-c2ba01412e81 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:18.272155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f89c1-5212-41c8-88f7-c2ba01412e81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:18.272155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f89c1-5212-41c8-88f7-c2ba01412e81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:18.272520 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f89c1-5212-41c8-88f7-c2ba01412e81 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:18.280382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 573/1150] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:07:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:18.286458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430e86f-bcd6-44c2-a1d0-934e5642aeac None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:18.347145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:18.388680 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430e86f-bcd6-44c2-a1d0-934e5642aeac None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:18.422145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bae342-0691-4e97-8d88-1754f8630a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:18.422846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bae342-0691-4e97-8d88-1754f8630a72 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:18.423171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bae342-0691-4e97-8d88-1754f8630a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:18.423448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bae342-0691-4e97-8d88-1754f8630a72 None admin] PATH_INFO: `/v3/users/7e4391c9e4604e029530ed49a6723012` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:18.424577 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bae342-0691-4e97-8d88-1754f8630a72 None admin] RBAC: Authorizing `identity:get_user(user_id=7e4391c9e4604e029530ed49a6723012)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:18.426005 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bae342-0691-4e97-8d88-1754f8630a72 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:18.427639 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 578/1151] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:07:18 2022] GET /identity/v3/users/7e4391c9e4604e029530ed49a6723012 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:18.433444 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f89c1-5212-41c8-88f7-c2ba01412e81 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:18.532728 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f89c1-5212-41c8-88f7-c2ba01412e81 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:18.565098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e92f2-5623-43bf-82e7-7d0101cc8c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:18.565717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e92f2-5623-43bf-82e7-7d0101cc8c75 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:18.565992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e92f2-5623-43bf-82e7-7d0101cc8c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:18.566264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e92f2-5623-43bf-82e7-7d0101cc8c75 None admin] PATH_INFO: `/v3/projects/dcdb536cf9ab436cbd0169a8eca319fd` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:18.567384 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e92f2-5623-43bf-82e7-7d0101cc8c75 None admin] RBAC: Authorizing `identity:get_project(project_id=dcdb536cf9ab436cbd0169a8eca319fd)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:18.568789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e92f2-5623-43bf-82e7-7d0101cc8c75 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:18.570389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 574/1152] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:07:18 2022] GET /identity/v3/projects/dcdb536cf9ab436cbd0169a8eca319fd => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:18.576380 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bae342-0691-4e97-8d88-1754f8630a72 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:18.675100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bae342-0691-4e97-8d88-1754f8630a72 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:18.701176 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:18.709439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3a916-08ad-4590-a04c-61f91c2c46ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:18.710097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3a916-08ad-4590-a04c-61f91c2c46ce None admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:18.710383 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3a916-08ad-4590-a04c-61f91c2c46ce None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:18.710656 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3a916-08ad-4590-a04c-61f91c2c46ce None admin] PATH_INFO: `/v3/projects/dcdb536cf9ab436cbd0169a8eca319fd/users/7e4391c9e4604e029530ed49a6723012/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:18.712580 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3a916-08ad-4590-a04c-61f91c2c46ce None admin] RBAC: Authorizing `identity:create_grant(project_id=dcdb536cf9ab436cbd0169a8eca319fd, user_id=7e4391c9e4604e029530ed49a6723012, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:18.719918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3a916-08ad-4590-a04c-61f91c2c46ce None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:18.818849 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 579/1153] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:07:18 2022] PUT /identity/v3/projects/dcdb536cf9ab436cbd0169a8eca319fd/users/7e4391c9e4604e029530ed49a6723012/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:07:20.720189 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be48d4-57ad-4d21-898c-f91cd2e5f6bd None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:20.720727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be48d4-57ad-4d21-898c-f91cd2e5f6bd None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:20.720727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be48d4-57ad-4d21-898c-f91cd2e5f6bd None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:20.722131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 575/1154] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:20.727240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd496b-ff9c-4496-9b14-c30d8e4b3e56 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:20.727655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd496b-ff9c-4496-9b14-c30d8e4b3e56 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:20.727793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd496b-ff9c-4496-9b14-c30d8e4b3e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:20.745945 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-73bd496b-ff9c-4496-9b14-c30d8e4b3e56 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:20.747159 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-73bd496b-ff9c-4496-9b14-c30d8e4b3e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:20.813716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 580/1155] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:21.087192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be48d4-57ad-4d21-898c-f91cd2e5f6bd None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:21.139494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:21.191030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be48d4-57ad-4d21-898c-f91cd2e5f6bd None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:21.223590 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345482c-40e3-4f1b-81ae-5089fbe9d800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:21.224215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345482c-40e3-4f1b-81ae-5089fbe9d800 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:21.224498 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345482c-40e3-4f1b-81ae-5089fbe9d800 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:21.224775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345482c-40e3-4f1b-81ae-5089fbe9d800 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:21.237048 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345482c-40e3-4f1b-81ae-5089fbe9d800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:07:21.241779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-4345482c-40e3-4f1b-81ae-5089fbe9d800 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:21.241779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:21.241779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:21.241779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:21.241779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:21.241779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:21.241779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:21.241779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:21.241779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:21.241779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:21.241779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:21.241779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:21.241779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:07:21.241779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:07:21.241779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:07:21.241779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:07:21.242926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:07:21.242926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:07:21.242926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:07:21.242926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:07:21.242926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:21.242926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:21.242926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:21.242926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:21.242926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:21.242926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:21.242926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:21.242926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:21.242926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:21.242926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:21.242926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:21.242926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:21.244016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:21.244016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:21.244016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:21.244016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:21.244016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:07:21.244016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:07:21.244016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:07:21.244016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:07:21.244016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:07:21.244016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:07:21.244016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:21.244016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:07:21.244016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 576/1156] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:07:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:21.249147 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd496b-ff9c-4496-9b14-c30d8e4b3e56 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:21.349388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd496b-ff9c-4496-9b14-c30d8e4b3e56 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:21.367323 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:21.384746 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c6349-fd21-419c-9698-97369e1230fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:21.385392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c6349-fd21-419c-9698-97369e1230fb None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:21.385670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c6349-fd21-419c-9698-97369e1230fb None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:21.385941 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c6349-fd21-419c-9698-97369e1230fb None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:21.386601 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c6349-fd21-419c-9698-97369e1230fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:21.386919 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c6349-fd21-419c-9698-97369e1230fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:21.388276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c6349-fd21-419c-9698-97369e1230fb None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:21.396356 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 581/1157] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:07:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:21.402489 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345482c-40e3-4f1b-81ae-5089fbe9d800 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:21.504044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345482c-40e3-4f1b-81ae-5089fbe9d800 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:21.514026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:21.539926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca6577-552c-42ee-a4d0-36026f6e446c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:21.540545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca6577-552c-42ee-a4d0-36026f6e446c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:21.540821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca6577-552c-42ee-a4d0-36026f6e446c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:21.541089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca6577-552c-42ee-a4d0-36026f6e446c None admin] PATH_INFO: `/v3/users/7e4391c9e4604e029530ed49a6723012` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:21.542187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca6577-552c-42ee-a4d0-36026f6e446c None admin] RBAC: Authorizing `identity:get_user(user_id=7e4391c9e4604e029530ed49a6723012)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:21.543630 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca6577-552c-42ee-a4d0-36026f6e446c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:21.545209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 577/1158] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:07:21 2022] GET /identity/v3/users/7e4391c9e4604e029530ed49a6723012 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:21.551256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c6349-fd21-419c-9698-97369e1230fb None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:21.649241 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c6349-fd21-419c-9698-97369e1230fb None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:21.682540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cff75a-f313-43c7-b54c-e940368ec156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:21.683172 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cff75a-f313-43c7-b54c-e940368ec156 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:21.683445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cff75a-f313-43c7-b54c-e940368ec156 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:21.683712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cff75a-f313-43c7-b54c-e940368ec156 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:21.730058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:21.750417 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cff75a-f313-43c7-b54c-e940368ec156 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:21.751875 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cff75a-f313-43c7-b54c-e940368ec156 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:21.846811 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 582/1159] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:07:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:07:21.852732 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca6577-552c-42ee-a4d0-36026f6e446c None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:21.949957 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca6577-552c-42ee-a4d0-36026f6e446c None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:21.983801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4a199-8e1b-441d-93fb-b7d690bf8069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:21.984468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4a199-8e1b-441d-93fb-b7d690bf8069 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:21.985009 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4a199-8e1b-441d-93fb-b7d690bf8069 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:21.985285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4a199-8e1b-441d-93fb-b7d690bf8069 None admin] PATH_INFO: `/v3/projects/dcdb536cf9ab436cbd0169a8eca319fd` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:21.986403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4a199-8e1b-441d-93fb-b7d690bf8069 None admin] RBAC: Authorizing `identity:get_project(project_id=dcdb536cf9ab436cbd0169a8eca319fd)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:21.987863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4a199-8e1b-441d-93fb-b7d690bf8069 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:21.989490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 578/1160] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:07:21 2022] GET /identity/v3/projects/dcdb536cf9ab436cbd0169a8eca319fd => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:21.995772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cff75a-f313-43c7-b54c-e940368ec156 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:22.079244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:22.098479 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cff75a-f313-43c7-b54c-e940368ec156 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:22.133784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc554377-2c5f-4b00-940b-f803ee00a53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:22.134392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc554377-2c5f-4b00-940b-f803ee00a53a None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:22.134676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc554377-2c5f-4b00-940b-f803ee00a53a None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:22.134984 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc554377-2c5f-4b00-940b-f803ee00a53a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:22.135714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc554377-2c5f-4b00-940b-f803ee00a53a None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=dcdb536cf9ab436cbd0169a8eca319fd, user.id=7e4391c9e4604e029530ed49a6723012) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:22.136027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc554377-2c5f-4b00-940b-f803ee00a53a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:22.137462 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc554377-2c5f-4b00-940b-f803ee00a53a None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:22.143817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 583/1161] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:07:21 2022] GET /identity/v3/role_assignments?user.id=7e4391c9e4604e029530ed49a6723012&scope.project.id=dcdb536cf9ab436cbd0169a8eca319fd&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:24.063510 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe32f7-17ed-46af-9a56-7b448de5def7 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:24.064083 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe32f7-17ed-46af-9a56-7b448de5def7 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:24.064083 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe32f7-17ed-46af-9a56-7b448de5def7 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:24.065393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 579/1162] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:24.070502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12edae8-726f-4d3c-b8ac-40f42cc907db None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:24.070955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12edae8-726f-4d3c-b8ac-40f42cc907db None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:24.071087 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12edae8-726f-4d3c-b8ac-40f42cc907db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:24.089386 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-f12edae8-726f-4d3c-b8ac-40f42cc907db None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:24.090341 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f12edae8-726f-4d3c-b8ac-40f42cc907db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:24.155708 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 584/1163] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:24.425563 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe32f7-17ed-46af-9a56-7b448de5def7 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:24.473505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:24.526220 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe32f7-17ed-46af-9a56-7b448de5def7 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:24.558714 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a00460-51b2-4019-91a6-b14e5efa460a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:24.559371 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a00460-51b2-4019-91a6-b14e5efa460a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:24.559648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a00460-51b2-4019-91a6-b14e5efa460a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:24.559916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a00460-51b2-4019-91a6-b14e5efa460a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:24.627088 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a00460-51b2-4019-91a6-b14e5efa460a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:24.628535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a00460-51b2-4019-91a6-b14e5efa460a None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:24.674859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:24.728071 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 580/1164] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:07:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:07:24.734226 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12edae8-726f-4d3c-b8ac-40f42cc907db None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:24.806169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:24.830685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12edae8-726f-4d3c-b8ac-40f42cc907db None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:24.864090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb200f56-cc6f-4f2c-b46d-f828bfe56476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:24.864710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb200f56-cc6f-4f2c-b46d-f828bfe56476 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:24.864986 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb200f56-cc6f-4f2c-b46d-f828bfe56476 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:24.865256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb200f56-cc6f-4f2c-b46d-f828bfe56476 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:24.871703 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb200f56-cc6f-4f2c-b46d-f828bfe56476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:24.873135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb200f56-cc6f-4f2c-b46d-f828bfe56476 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:24.884871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-cb200f56-cc6f-4f2c-b46d-f828bfe56476 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 19:07:24.884871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:24.884871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 31 19:07:24.884871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 31 19:07:24.884871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:07:24.884871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:07:24.884871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:07:24.884871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:07:24.884871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 31 19:07:24.884871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:07:24.884871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:07:24.884871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:07:24.884871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:24.884871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:24.884871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:24.884871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:24.884871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:24.886170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:24.886170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:24.886170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:24.886170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:24.886170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:24.886170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:24.886170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:07:24.886170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:07:24.886170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:07:24.886170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:07:24.886170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:24.886170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:24.886170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:24.886170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:24.886170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:24.886170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:24.887478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:24.887478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:24.887478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:24.887478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:24.887478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:24.887478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:24.887478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:24.887478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:24.887478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:24.887478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:24.887478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 31 19:07:24.887478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:07:24.887478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 19:07:24.887478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:07:24.887478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 585/1165] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 31 19:07:24 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:24.892650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a00460-51b2-4019-91a6-b14e5efa460a None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:24.990421 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a00460-51b2-4019-91a6-b14e5efa460a None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:25.025990 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe8593-23ac-4cac-85cf-c71bc295f7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:25.026596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe8593-23ac-4cac-85cf-c71bc295f7ba None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:25.026898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe8593-23ac-4cac-85cf-c71bc295f7ba None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:25.027167 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe8593-23ac-4cac-85cf-c71bc295f7ba None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:25.027822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe8593-23ac-4cac-85cf-c71bc295f7ba None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:25.028121 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe8593-23ac-4cac-85cf-c71bc295f7ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:25.029468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe8593-23ac-4cac-85cf-c71bc295f7ba None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:25.037359 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 581/1166] 10.222.0.59 () {56 vars in 1115 bytes} [Wed Aug 31 19:07:24 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:26.958729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaf922-c784-44a1-bf5c-2f62dfa93900 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:26.959308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaf922-c784-44a1-bf5c-2f62dfa93900 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:26.959308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaf922-c784-44a1-bf5c-2f62dfa93900 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:26.960649 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 586/1167] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:26.965849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93871366-dcf1-413f-813d-69f97c259607 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:26.966231 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93871366-dcf1-413f-813d-69f97c259607 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:26.966411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93871366-dcf1-413f-813d-69f97c259607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:26.984838 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-93871366-dcf1-413f-813d-69f97c259607 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:26.986032 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-93871366-dcf1-413f-813d-69f97c259607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:26.988803 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:27.056307 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 582/1168] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:27.329270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afaf922-c784-44a1-bf5c-2f62dfa93900 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:27.433126 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afaf922-c784-44a1-bf5c-2f62dfa93900 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:27.466961 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2424c-36b8-458e-a33d-36cba73e501b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:27.467568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2424c-36b8-458e-a33d-36cba73e501b None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:27.467855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2424c-36b8-458e-a33d-36cba73e501b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:27.468127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2424c-36b8-458e-a33d-36cba73e501b None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:27.469329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2424c-36b8-458e-a33d-36cba73e501b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:27.471027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2424c-36b8-458e-a33d-36cba73e501b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:27.474903 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:27.544417 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 587/1169] 10.222.0.59 () {60 vars in 1138 bytes} [Wed Aug 31 19:07:27 2022] POST /identity/v3/domains => generated 267 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:07:29.471381 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0790bb-6db1-4d15-a05a-50bb4230cb6d None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:29.472030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0790bb-6db1-4d15-a05a-50bb4230cb6d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:29.472030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0790bb-6db1-4d15-a05a-50bb4230cb6d None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:29.473275 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 583/1170] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:29.478654 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168feb6-612c-452f-ab26-bc579784ef1a None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:29.479354 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168feb6-612c-452f-ab26-bc579784ef1a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:29.479614 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168feb6-612c-452f-ab26-bc579784ef1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:29.498085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-3168feb6-612c-452f-ab26-bc579784ef1a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:29.499374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3168feb6-612c-452f-ab26-bc579784ef1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:29.565139 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 588/1171] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:29.832808 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0790bb-6db1-4d15-a05a-50bb4230cb6d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:29.933784 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0790bb-6db1-4d15-a05a-50bb4230cb6d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:29.968638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcccb973-95e6-44ca-ada0-feda1796f8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:29.969516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcccb973-95e6-44ca-ada0-feda1796f8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:29.969794 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcccb973-95e6-44ca-ada0-feda1796f8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:29.970065 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcccb973-95e6-44ca-ada0-feda1796f8b1 None admin] PATH_INFO: `/v3/domains/5d9759b53ce3427d94d2aa3aa6f06faa` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:29.978139 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:29.980580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcccb973-95e6-44ca-ada0-feda1796f8b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5d9759b53ce3427d94d2aa3aa6f06faa)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:29.982004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcccb973-95e6-44ca-ada0-feda1796f8b1 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:29.983625 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 584/1172] 10.222.0.59 () {56 vars in 1183 bytes} [Wed Aug 31 19:07:29 2022] GET /identity/v3/domains/5d9759b53ce3427d94d2aa3aa6f06faa => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:29.989852 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168feb6-612c-452f-ab26-bc579784ef1a None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:30.088913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168feb6-612c-452f-ab26-bc579784ef1a None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:30.112391 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:30.123076 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ebc43-0b10-4e84-b091-8d6f1f32a6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:30.123685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ebc43-0b10-4e84-b091-8d6f1f32a6ef None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:30.123967 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ebc43-0b10-4e84-b091-8d6f1f32a6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:30.128423 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ebc43-0b10-4e84-b091-8d6f1f32a6ef None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:30.128423 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ebc43-0b10-4e84-b091-8d6f1f32a6ef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:30.128423 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ebc43-0b10-4e84-b091-8d6f1f32a6ef None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:30.236245 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 589/1173] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 31 19:07:29 2022] POST /identity/v3/projects => generated 361 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:07:32.208319 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0eb62-654e-45ad-9356-2d94e750704f None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:32.208916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0eb62-654e-45ad-9356-2d94e750704f None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:32.208916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0eb62-654e-45ad-9356-2d94e750704f None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:32.210247 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 585/1174] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:32.215852 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab137d-0246-406e-af40-82eb7309604b None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:32.216276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab137d-0246-406e-af40-82eb7309604b None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:32.216406 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab137d-0246-406e-af40-82eb7309604b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:32.235230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-77ab137d-0246-406e-af40-82eb7309604b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:32.236546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-77ab137d-0246-406e-af40-82eb7309604b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:32.305282 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 590/1175] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:32.583133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0eb62-654e-45ad-9356-2d94e750704f None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:32.693973 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0eb62-654e-45ad-9356-2d94e750704f None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:32.728767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89ebcb-f131-451d-8394-ea0217c7a04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:32.729397 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89ebcb-f131-451d-8394-ea0217c7a04c None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:32.729669 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89ebcb-f131-451d-8394-ea0217c7a04c None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:32.729938 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89ebcb-f131-451d-8394-ea0217c7a04c None admin] PATH_INFO: `/v3/domains/5d9759b53ce3427d94d2aa3aa6f06faa` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:32.731119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89ebcb-f131-451d-8394-ea0217c7a04c None admin] RBAC: Authorizing `identity:get_domain(domain_id=5d9759b53ce3427d94d2aa3aa6f06faa)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:32.732537 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89ebcb-f131-451d-8394-ea0217c7a04c None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:32.734107 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 586/1176] 10.222.0.59 () {56 vars in 1183 bytes} [Wed Aug 31 19:07:32 2022] GET /identity/v3/domains/5d9759b53ce3427d94d2aa3aa6f06faa => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:32.740299 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab137d-0246-406e-af40-82eb7309604b None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:32.841835 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab137d-0246-406e-af40-82eb7309604b None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:32.849283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:32.875857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d790b96-c7a3-4be5-8845-f5ddb3e4ac43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:32.876463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d790b96-c7a3-4be5-8845-f5ddb3e4ac43 None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:32.876744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d790b96-c7a3-4be5-8845-f5ddb3e4ac43 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:32.877013 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d790b96-c7a3-4be5-8845-f5ddb3e4ac43 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:32.877677 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d790b96-c7a3-4be5-8845-f5ddb3e4ac43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:32.879129 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d790b96-c7a3-4be5-8845-f5ddb3e4ac43 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:32.965851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 591/1177] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:07:32 2022] POST /identity/v3/users => generated 311 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:07:34.940979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bcc63-ab94-4140-b82a-4bc2361f92d2 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:34.941524 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bcc63-ab94-4140-b82a-4bc2361f92d2 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:34.941524 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bcc63-ab94-4140-b82a-4bc2361f92d2 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:34.942862 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 587/1178] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:34.948394 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e745e-5d8d-4844-ba2b-bc24b7c64636 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:34.948837 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e745e-5d8d-4844-ba2b-bc24b7c64636 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:34.948940 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e745e-5d8d-4844-ba2b-bc24b7c64636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:34.967822 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-ec3e745e-5d8d-4844-ba2b-bc24b7c64636 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:34.969642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ec3e745e-5d8d-4844-ba2b-bc24b7c64636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:35.042406 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 592/1179] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:35.317217 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bcc63-ab94-4140-b82a-4bc2361f92d2 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:35.387830 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:35.426861 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bcc63-ab94-4140-b82a-4bc2361f92d2 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:35.462044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a44762-2e43-4533-931f-4c022d1f7104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:35.462687 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a44762-2e43-4533-931f-4c022d1f7104 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:35.462992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a44762-2e43-4533-931f-4c022d1f7104 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:35.463268 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a44762-2e43-4533-931f-4c022d1f7104 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:35.474307 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a44762-2e43-4533-931f-4c022d1f7104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:07:35.479112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-97a44762-2e43-4533-931f-4c022d1f7104 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:35.479112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:35.479112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:35.479112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:35.479112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:35.479112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:35.479112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:35.479112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:35.479112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:35.479112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:35.479112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:35.479112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:35.479112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:07:35.479112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:07:35.479112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:07:35.479112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:07:35.480375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:07:35.480375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:07:35.480375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:07:35.480375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:07:35.480375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:35.480375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:35.480375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:35.480375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:35.480375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:35.480375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:35.480375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:35.480375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:35.480375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:35.480375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:35.480375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:35.480375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:35.481528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:35.481528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:35.481528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:35.481528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:35.481528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:07:35.481528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:07:35.481528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:07:35.481528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:07:35.481528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:07:35.481528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:07:35.481528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:35.481528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:07:35.481528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 588/1180] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:07:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:35.487774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e745e-5d8d-4844-ba2b-bc24b7c64636 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:35.580006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:35.588718 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e745e-5d8d-4844-ba2b-bc24b7c64636 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:35.623585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de457781-16ce-4d7f-87e7-3db25338b062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:35.624295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de457781-16ce-4d7f-87e7-3db25338b062 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:35.624663 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de457781-16ce-4d7f-87e7-3db25338b062 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:35.625628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de457781-16ce-4d7f-87e7-3db25338b062 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:35.626309 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de457781-16ce-4d7f-87e7-3db25338b062 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:35.626617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de457781-16ce-4d7f-87e7-3db25338b062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:35.628072 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de457781-16ce-4d7f-87e7-3db25338b062 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:35.636829 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 593/1181] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:07:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:35.642674 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a44762-2e43-4533-931f-4c022d1f7104 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:35.744073 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a44762-2e43-4533-931f-4c022d1f7104 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:35.772005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:35.786509 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2fdef-bf09-4148-8672-1cf0469332b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:35.789970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2fdef-bf09-4148-8672-1cf0469332b4 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:35.789970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2fdef-bf09-4148-8672-1cf0469332b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:35.789970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2fdef-bf09-4148-8672-1cf0469332b4 None admin] PATH_INFO: `/v3/users/bf85a4fe6ead4d38a3ae928d21a1439e` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:35.801581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea2fdef-bf09-4148-8672-1cf0469332b4 None admin] RBAC: Authorizing `identity:get_user(user_id=bf85a4fe6ead4d38a3ae928d21a1439e)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:35.803058 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea2fdef-bf09-4148-8672-1cf0469332b4 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:35.804712 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 589/1182] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:07:35 2022] GET /identity/v3/users/bf85a4fe6ead4d38a3ae928d21a1439e => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:35.811021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de457781-16ce-4d7f-87e7-3db25338b062 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:35.910504 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de457781-16ce-4d7f-87e7-3db25338b062 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:35.944042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10cce0-82be-4f5c-92f9-1c75090b389c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:35.944652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10cce0-82be-4f5c-92f9-1c75090b389c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:35.944935 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10cce0-82be-4f5c-92f9-1c75090b389c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:35.945211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10cce0-82be-4f5c-92f9-1c75090b389c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:35.969598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:36.017136 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10cce0-82be-4f5c-92f9-1c75090b389c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:36.018591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10cce0-82be-4f5c-92f9-1c75090b389c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:36.120152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 594/1183] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:07:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:07:36.126817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2fdef-bf09-4148-8672-1cf0469332b4 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:36.231107 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2fdef-bf09-4148-8672-1cf0469332b4 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:36.266287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48dac48-e813-4c96-b291-70509fc80d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:36.267018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dac48-e813-4c96-b291-70509fc80d72 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:36.267331 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dac48-e813-4c96-b291-70509fc80d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:36.267635 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dac48-e813-4c96-b291-70509fc80d72 None admin] PATH_INFO: `/v3/projects/c02171771c3141a0b1b27192a4576f32` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:36.268781 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48dac48-e813-4c96-b291-70509fc80d72 None admin] RBAC: Authorizing `identity:get_project(project_id=c02171771c3141a0b1b27192a4576f32)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:36.270213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48dac48-e813-4c96-b291-70509fc80d72 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:36.272004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 590/1184] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:07:36 2022] GET /identity/v3/projects/c02171771c3141a0b1b27192a4576f32 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:36.278144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10cce0-82be-4f5c-92f9-1c75090b389c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:36.336888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:36.382435 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10cce0-82be-4f5c-92f9-1c75090b389c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:36.416954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ea1e3-a9eb-4357-a198-659074e4682c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:36.417558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ea1e3-a9eb-4357-a198-659074e4682c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:36.417842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ea1e3-a9eb-4357-a198-659074e4682c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:36.418118 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ea1e3-a9eb-4357-a198-659074e4682c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:36.418847 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ea1e3-a9eb-4357-a198-659074e4682c None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=c02171771c3141a0b1b27192a4576f32, user.id=bf85a4fe6ead4d38a3ae928d21a1439e) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:36.419277 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ea1e3-a9eb-4357-a198-659074e4682c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:36.420710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ea1e3-a9eb-4357-a198-659074e4682c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:36.426921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 595/1185] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:07:36 2022] GET /identity/v3/role_assignments?user.id=bf85a4fe6ead4d38a3ae928d21a1439e&scope.project.id=c02171771c3141a0b1b27192a4576f32&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:38.316698 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc249030-e919-417e-9e19-17a2c7d5e103 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:38.317270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc249030-e919-417e-9e19-17a2c7d5e103 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:38.317270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc249030-e919-417e-9e19-17a2c7d5e103 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:38.318585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 591/1186] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:38.323980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b636a97-e7b9-401e-9ef9-343556ec5ec4 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:38.324410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b636a97-e7b9-401e-9ef9-343556ec5ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:38.324529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b636a97-e7b9-401e-9ef9-343556ec5ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:38.343095 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-6b636a97-e7b9-401e-9ef9-343556ec5ec4 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:38.344272 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6b636a97-e7b9-401e-9ef9-343556ec5ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:38.410074 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 596/1187] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:38.679841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc249030-e919-417e-9e19-17a2c7d5e103 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:38.738515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:38.784489 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc249030-e919-417e-9e19-17a2c7d5e103 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:38.817730 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe8e2e-2831-4618-9ccd-264c3170a517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:38.818345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe8e2e-2831-4618-9ccd-264c3170a517 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:38.818619 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe8e2e-2831-4618-9ccd-264c3170a517 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:38.818910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe8e2e-2831-4618-9ccd-264c3170a517 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:38.828932 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe8e2e-2831-4618-9ccd-264c3170a517 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:07:38.833634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-e5fe8e2e-2831-4618-9ccd-264c3170a517 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:38.833634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:38.833634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:38.833634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:38.833634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:38.833634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:38.833634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:38.833634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:38.833634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:38.833634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:38.833634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:38.833634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:38.833634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:07:38.833634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:07:38.833634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:07:38.833634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:07:38.839042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:07:38.839042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:07:38.839042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:07:38.839042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:07:38.839042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:38.839042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:38.839042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:38.839042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:38.839042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:38.839042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:38.839042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:38.839042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:38.839042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:38.839042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:38.839042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:38.839042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:38.840102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:38.840102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:38.840102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:38.840102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:38.840102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:07:38.840102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:07:38.840102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:07:38.840102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:07:38.840102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:07:38.840102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:07:38.840102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:38.840102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:07:38.840102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 592/1188] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:07:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:38.840867 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b636a97-e7b9-401e-9ef9-343556ec5ec4 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:38.892748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:38.944369 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b636a97-e7b9-401e-9ef9-343556ec5ec4 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:38.978673 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6bcdd-2952-4a0f-ad8a-bd36e5e52234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:38.979307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6bcdd-2952-4a0f-ad8a-bd36e5e52234 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:38.979593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6bcdd-2952-4a0f-ad8a-bd36e5e52234 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:38.979866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6bcdd-2952-4a0f-ad8a-bd36e5e52234 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:38.980524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6bcdd-2952-4a0f-ad8a-bd36e5e52234 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:38.980828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6bcdd-2952-4a0f-ad8a-bd36e5e52234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:38.982184 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6bcdd-2952-4a0f-ad8a-bd36e5e52234 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:38.991183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 597/1189] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:07:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:38.997198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe8e2e-2831-4618-9ccd-264c3170a517 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:39.097769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe8e2e-2831-4618-9ccd-264c3170a517 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:39.112896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:39.133195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061392c0-61d9-4460-9010-6e85ef950e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:39.133817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061392c0-61d9-4460-9010-6e85ef950e87 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:39.134096 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061392c0-61d9-4460-9010-6e85ef950e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:39.134368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061392c0-61d9-4460-9010-6e85ef950e87 None admin] PATH_INFO: `/v3/users/bf85a4fe6ead4d38a3ae928d21a1439e` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:39.135496 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061392c0-61d9-4460-9010-6e85ef950e87 None admin] RBAC: Authorizing `identity:get_user(user_id=bf85a4fe6ead4d38a3ae928d21a1439e)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:39.136968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061392c0-61d9-4460-9010-6e85ef950e87 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:39.138551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 593/1190] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:07:38 2022] GET /identity/v3/users/bf85a4fe6ead4d38a3ae928d21a1439e => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:39.145167 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6bcdd-2952-4a0f-ad8a-bd36e5e52234 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:39.244922 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6bcdd-2952-4a0f-ad8a-bd36e5e52234 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:39.255814 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:39.279712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531276d-0292-465d-b250-e5f0af07bf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:39.280352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531276d-0292-465d-b250-e5f0af07bf1a None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:39.280634 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531276d-0292-465d-b250-e5f0af07bf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:39.281213 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531276d-0292-465d-b250-e5f0af07bf1a None admin] PATH_INFO: `/v3/projects/c02171771c3141a0b1b27192a4576f32` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:39.282573 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a531276d-0292-465d-b250-e5f0af07bf1a None admin] RBAC: Authorizing `identity:get_project(project_id=c02171771c3141a0b1b27192a4576f32)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:39.284091 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a531276d-0292-465d-b250-e5f0af07bf1a None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:39.285764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 598/1191] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:07:39 2022] GET /identity/v3/projects/c02171771c3141a0b1b27192a4576f32 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:39.291693 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061392c0-61d9-4460-9010-6e85ef950e87 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:39.393649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061392c0-61d9-4460-9010-6e85ef950e87 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:39.427556 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a946a-efed-43fc-9040-c262aa9beb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:39.428216 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a946a-efed-43fc-9040-c262aa9beb09 None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:39.428492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a946a-efed-43fc-9040-c262aa9beb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:39.428762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a946a-efed-43fc-9040-c262aa9beb09 None admin] PATH_INFO: `/v3/projects/c02171771c3141a0b1b27192a4576f32/users/bf85a4fe6ead4d38a3ae928d21a1439e/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:39.430635 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a946a-efed-43fc-9040-c262aa9beb09 None admin] RBAC: Authorizing `identity:create_grant(project_id=c02171771c3141a0b1b27192a4576f32, user_id=bf85a4fe6ead4d38a3ae928d21a1439e, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:39.432107 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a946a-efed-43fc-9040-c262aa9beb09 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:39.509559 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 594/1192] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:07:39 2022] PUT /identity/v3/projects/c02171771c3141a0b1b27192a4576f32/users/bf85a4fe6ead4d38a3ae928d21a1439e/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:07:41.385670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82414db5-6343-4d10-aebf-c6d9ed7e7335 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:41.386218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82414db5-6343-4d10-aebf-c6d9ed7e7335 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:41.386218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82414db5-6343-4d10-aebf-c6d9ed7e7335 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:41.387599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 599/1193] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:41.393345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a53e5-ed4a-403e-abf2-e420cd906551 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:41.393776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a53e5-ed4a-403e-abf2-e420cd906551 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:41.393898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a53e5-ed4a-403e-abf2-e420cd906551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:41.413101 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-de7a53e5-ed4a-403e-abf2-e420cd906551 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:41.414294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-de7a53e5-ed4a-403e-abf2-e420cd906551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:41.449795 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:41.481082 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 595/1194] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:41.746049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82414db5-6343-4d10-aebf-c6d9ed7e7335 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:41.847110 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82414db5-6343-4d10-aebf-c6d9ed7e7335 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:41.880734 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff7e7c-dcbf-4018-a981-b0fe1c85537e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:41.881361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff7e7c-dcbf-4018-a981-b0fe1c85537e None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:41.881643 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff7e7c-dcbf-4018-a981-b0fe1c85537e None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:41.881918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff7e7c-dcbf-4018-a981-b0fe1c85537e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:41.893101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff7e7c-dcbf-4018-a981-b0fe1c85537e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:07:41.897884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-2cff7e7c-dcbf-4018-a981-b0fe1c85537e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:41.897884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:41.897884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:41.897884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:41.897884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:41.897884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:41.897884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:41.897884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:41.897884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:41.897884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:41.897884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:41.897884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:41.897884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:07:41.897884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:07:41.897884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:07:41.897884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:07:41.899055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:07:41.899055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:07:41.899055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:07:41.899055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:07:41.899055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:41.899055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:41.899055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:41.899055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:41.899055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:41.899055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:41.899055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:41.899055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:41.899055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:41.899055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:41.899055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:41.899055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:41.899931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:41.899931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:41.899931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:41.899931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:41.899931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:07:41.899931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:07:41.899931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:07:41.899931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:07:41.899931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:07:41.899931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:07:41.899931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:41.899931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:07:41.899931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 600/1195] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:07:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:41.905201 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a53e5-ed4a-403e-abf2-e420cd906551 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:42.011797 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a53e5-ed4a-403e-abf2-e420cd906551 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:42.045880 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed103b-3cd2-4157-9d36-b80218bb64f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:42.046483 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed103b-3cd2-4157-9d36-b80218bb64f0 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:42.046770 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed103b-3cd2-4157-9d36-b80218bb64f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:42.047077 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed103b-3cd2-4157-9d36-b80218bb64f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:42.047732 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed103b-3cd2-4157-9d36-b80218bb64f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:42.048038 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed103b-3cd2-4157-9d36-b80218bb64f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:42.049416 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed103b-3cd2-4157-9d36-b80218bb64f0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:42.057430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 596/1196] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:07:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:42.064093 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff7e7c-dcbf-4018-a981-b0fe1c85537e None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:42.104385 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:42.163365 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff7e7c-dcbf-4018-a981-b0fe1c85537e None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:42.196143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d0011-c0f5-4f14-812b-44b3ee0bfab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:42.196764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d0011-c0f5-4f14-812b-44b3ee0bfab0 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:42.197038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d0011-c0f5-4f14-812b-44b3ee0bfab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:42.197307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d0011-c0f5-4f14-812b-44b3ee0bfab0 None admin] PATH_INFO: `/v3/users/bf85a4fe6ead4d38a3ae928d21a1439e` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:42.198398 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453d0011-c0f5-4f14-812b-44b3ee0bfab0 None admin] RBAC: Authorizing `identity:get_user(user_id=bf85a4fe6ead4d38a3ae928d21a1439e)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:42.199853 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453d0011-c0f5-4f14-812b-44b3ee0bfab0 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:42.201444 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 601/1197] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 31 19:07:42 2022] GET /identity/v3/users/bf85a4fe6ead4d38a3ae928d21a1439e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:07:42.207562 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed103b-3cd2-4157-9d36-b80218bb64f0 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:42.242572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:42.311131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed103b-3cd2-4157-9d36-b80218bb64f0 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:42.345141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c004b81-4831-46e9-adfb-47f6f65a6092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:42.345766 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c004b81-4831-46e9-adfb-47f6f65a6092 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:42.346041 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c004b81-4831-46e9-adfb-47f6f65a6092 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:42.346308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c004b81-4831-46e9-adfb-47f6f65a6092 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:42.412390 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c004b81-4831-46e9-adfb-47f6f65a6092 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:42.413818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c004b81-4831-46e9-adfb-47f6f65a6092 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:42.446358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:42.511373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 597/1198] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:07:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:07:42.517992 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d0011-c0f5-4f14-812b-44b3ee0bfab0 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:42.617768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d0011-c0f5-4f14-812b-44b3ee0bfab0 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:42.625165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:42.652437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec0526-9e54-49b6-89ec-6da7983c79d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:42.653169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec0526-9e54-49b6-89ec-6da7983c79d8 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:42.653451 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec0526-9e54-49b6-89ec-6da7983c79d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:42.653727 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec0526-9e54-49b6-89ec-6da7983c79d8 None admin] PATH_INFO: `/v3/projects/c02171771c3141a0b1b27192a4576f32` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:42.654855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec0526-9e54-49b6-89ec-6da7983c79d8 None admin] RBAC: Authorizing `identity:get_project(project_id=c02171771c3141a0b1b27192a4576f32)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:42.656575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec0526-9e54-49b6-89ec-6da7983c79d8 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:42.658708 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 602/1199] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 31 19:07:42 2022] GET /identity/v3/projects/c02171771c3141a0b1b27192a4576f32 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:42.664577 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c004b81-4831-46e9-adfb-47f6f65a6092 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:42.763041 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c004b81-4831-46e9-adfb-47f6f65a6092 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:42.796725 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31293cb-326e-4e20-b0ea-cbc2018e9217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:42.797316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31293cb-326e-4e20-b0ea-cbc2018e9217 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:42.797596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31293cb-326e-4e20-b0ea-cbc2018e9217 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:42.797867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31293cb-326e-4e20-b0ea-cbc2018e9217 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:42.798578 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31293cb-326e-4e20-b0ea-cbc2018e9217 None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=c02171771c3141a0b1b27192a4576f32, user.id=bf85a4fe6ead4d38a3ae928d21a1439e) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:42.798915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31293cb-326e-4e20-b0ea-cbc2018e9217 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:42.800337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31293cb-326e-4e20-b0ea-cbc2018e9217 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:42.802086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:42.806530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 598/1200] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:07:42 2022] GET /identity/v3/role_assignments?user.id=bf85a4fe6ead4d38a3ae928d21a1439e&scope.project.id=c02171771c3141a0b1b27192a4576f32&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:52.659285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=13 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:52.834534 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:54.469167 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92d6f0-b8bf-47a8-a4f5-f6e614beb644 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:54.469799 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92d6f0-b8bf-47a8-a4f5-f6e614beb644 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:54.469799 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92d6f0-b8bf-47a8-a4f5-f6e614beb644 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:54.471114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 603/1201] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:08:54.476922 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f925197-ca38-43a7-9001-0e0c5013616b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:54.477767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f925197-ca38-43a7-9001-0e0c5013616b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:54.477767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f925197-ca38-43a7-9001-0e0c5013616b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:54.496633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-5f925197-ca38-43a7-9001-0e0c5013616b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:08:54.497834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-5f925197-ca38-43a7-9001-0e0c5013616b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:54.565192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 599/1202] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:08:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:08:54.841432 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92d6f0-b8bf-47a8-a4f5-f6e614beb644 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:54.946261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92d6f0-b8bf-47a8-a4f5-f6e614beb644 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:54.980571 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a2377-776d-43c8-ba14-a20975ea6b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:54.981201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a2377-776d-43c8-ba14-a20975ea6b3c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:54.981479 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a2377-776d-43c8-ba14-a20975ea6b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:54.981744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a2377-776d-43c8-ba14-a20975ea6b3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:54.992874 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a2377-776d-43c8-ba14-a20975ea6b3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:54.994307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a2377-776d-43c8-ba14-a20975ea6b3c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:55.008695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-f92a2377-776d-43c8-ba14-a20975ea6b3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:08:55.008695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:08:55.008695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:08:55.008695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:08:55.008695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:08:55.008695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:08:55.008695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:08:55.008695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:08:55.008695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:08:55.008695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:08:55.008695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:08:55.008695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:08:55.008695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:08:55.008695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:08:55.008695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:08:55.008695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:08:55.009918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:08:55.009918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:08:55.009918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:08:55.009918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:08:55.009918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:08:55.009918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:08:55.009918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:08:55.009918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:08:55.009918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:08:55.009918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:08:55.009918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:08:55.009918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:08:55.009918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:08:55.009918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:08:55.009918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:08:55.009918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:08:55.009918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:08:55.011157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:08:55.011157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:08:55.011157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:08:55.011157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 604/1203] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:08:55.015942 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f925197-ca38-43a7-9001-0e0c5013616b None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:55.116494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f925197-ca38-43a7-9001-0e0c5013616b None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:55.145139 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:55.150848 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb9ef8-1116-40ce-b5db-d8d1c320ad76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:55.151484 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb9ef8-1116-40ce-b5db-d8d1c320ad76 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:55.151762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb9ef8-1116-40ce-b5db-d8d1c320ad76 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:55.152140 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb9ef8-1116-40ce-b5db-d8d1c320ad76 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:55.152706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb9ef8-1116-40ce-b5db-d8d1c320ad76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:08:55.153008 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb9ef8-1116-40ce-b5db-d8d1c320ad76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:55.154359 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb9ef8-1116-40ce-b5db-d8d1c320ad76 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:55.165883 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 600/1204] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:08:55.171812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a2377-776d-43c8-ba14-a20975ea6b3c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:55.225518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:55.271779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a2377-776d-43c8-ba14-a20975ea6b3c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:55.306418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4abde-f1f8-4e7d-a047-3936833c02d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:55.307074 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4abde-f1f8-4e7d-a047-3936833c02d4 None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:55.307524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4abde-f1f8-4e7d-a047-3936833c02d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:55.308313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4abde-f1f8-4e7d-a047-3936833c02d4 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:55.308981 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4abde-f1f8-4e7d-a047-3936833c02d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:55.310412 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4abde-f1f8-4e7d-a047-3936833c02d4 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:55.399749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 605/1205] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 31 19:08:55 2022] POST /identity/v3/users => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:08:57.339341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d35f3-22b2-4d6d-9d38-e928ec20f895 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:57.339874 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d35f3-22b2-4d6d-9d38-e928ec20f895 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:57.339874 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d35f3-22b2-4d6d-9d38-e928ec20f895 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:57.341264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 601/1206] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:08:57.346982 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74906a2-5fab-449e-81ac-a33394effbac None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:57.356360 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74906a2-5fab-449e-81ac-a33394effbac None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:57.356360 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74906a2-5fab-449e-81ac-a33394effbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:57.368844 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d74906a2-5fab-449e-81ac-a33394effbac None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:08:57.370308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d74906a2-5fab-449e-81ac-a33394effbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:57.445924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 606/1207] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:08:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:08:57.717317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d35f3-22b2-4d6d-9d38-e928ec20f895 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:57.823983 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d35f3-22b2-4d6d-9d38-e928ec20f895 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:57.864543 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef1d7b-c6c5-4b89-b112-8a170b4e8776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:57.865192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef1d7b-c6c5-4b89-b112-8a170b4e8776 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:57.865993 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef1d7b-c6c5-4b89-b112-8a170b4e8776 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:57.866262 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef1d7b-c6c5-4b89-b112-8a170b4e8776 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:57.878992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef1d7b-c6c5-4b89-b112-8a170b4e8776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:08:57.884516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-b8ef1d7b-c6c5-4b89-b112-8a170b4e8776 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:08:57.884516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:08:57.884516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:08:57.884516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:08:57.884516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:08:57.884516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:08:57.884516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:08:57.884516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:08:57.884516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:08:57.884516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:08:57.884516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:08:57.884516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:08:57.884516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:08:57.884516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:08:57.884516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:08:57.884516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:08:57.885608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:08:57.885608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:08:57.885608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:08:57.885608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:08:57.885608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:08:57.885608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:08:57.885608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:08:57.885608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:08:57.885608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:08:57.885608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:08:57.885608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:08:57.885608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:08:57.885608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:08:57.885608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:08:57.885608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:08:57.885608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:08:57.886502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:08:57.886502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:08:57.886502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:08:57.886502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:08:57.886502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:08:57.886502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:08:57.886502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:08:57.886502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:08:57.886502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:08:57.886502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:08:57.886502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:08:57.886502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:08:57.886502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 602/1208] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:08:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:08:57.893437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74906a2-5fab-449e-81ac-a33394effbac None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:57.933333 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:57.996795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74906a2-5fab-449e-81ac-a33394effbac None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:58.041531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8d266-fd0b-4edf-b1b5-00b772e99bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:58.042147 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8d266-fd0b-4edf-b1b5-00b772e99bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:58.042425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8d266-fd0b-4edf-b1b5-00b772e99bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:58.042695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8d266-fd0b-4edf-b1b5-00b772e99bb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:58.043419 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8d266-fd0b-4edf-b1b5-00b772e99bb5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:08:58.043720 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8d266-fd0b-4edf-b1b5-00b772e99bb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:58.045089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8d266-fd0b-4edf-b1b5-00b772e99bb5 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:58.053629 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 607/1209] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:08:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:08:58.065020 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef1d7b-c6c5-4b89-b112-8a170b4e8776 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:58.111244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:58.160650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef1d7b-c6c5-4b89-b112-8a170b4e8776 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:58.196626 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d43f43-4721-44bc-a25f-6d415ad0941b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:58.197268 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d43f43-4721-44bc-a25f-6d415ad0941b None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:58.197550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d43f43-4721-44bc-a25f-6d415ad0941b None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:58.197822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d43f43-4721-44bc-a25f-6d415ad0941b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:58.208726 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d43f43-4721-44bc-a25f-6d415ad0941b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:58.210342 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d43f43-4721-44bc-a25f-6d415ad0941b None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:58.225476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-a7d43f43-4721-44bc-a25f-6d415ad0941b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:08:58.225476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:08:58.225476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:08:58.225476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:08:58.225476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:08:58.225476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:08:58.225476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:08:58.225476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:08:58.225476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:08:58.225476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:08:58.225476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:08:58.225476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:08:58.225476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:08:58.225476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:08:58.225476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:08:58.225476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:08:58.226691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:08:58.226691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:08:58.226691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:08:58.226691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:08:58.226691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:08:58.226691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:08:58.226691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:08:58.226691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:08:58.226691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:08:58.226691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:08:58.226691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:08:58.226691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:08:58.226691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:08:58.226691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:08:58.226691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:08:58.226691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:08:58.226691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:08:58.228062 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:08:58.228062 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:08:58.228062 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:08:58.228062 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 603/1210] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:08:58.232991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8d266-fd0b-4edf-b1b5-00b772e99bb5 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:58.332151 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:58.335770 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8d266-fd0b-4edf-b1b5-00b772e99bb5 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:58.369511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d614b-21b8-4baf-995d-f9c17b6ba7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:58.370167 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d614b-21b8-4baf-995d-f9c17b6ba7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:58.370385 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d614b-21b8-4baf-995d-f9c17b6ba7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:58.370656 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d614b-21b8-4baf-995d-f9c17b6ba7a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:58.371331 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d614b-21b8-4baf-995d-f9c17b6ba7a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:08:58.371636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d614b-21b8-4baf-995d-f9c17b6ba7a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:58.373000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d614b-21b8-4baf-995d-f9c17b6ba7a3 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:58.385139 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 608/1211] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:08:58.391006 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d43f43-4721-44bc-a25f-6d415ad0941b None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:58.490169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d43f43-4721-44bc-a25f-6d415ad0941b None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:58.492347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:58.524970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32c6e5-039b-4e6e-8e6f-1ebd0de8fb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:58.525533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32c6e5-039b-4e6e-8e6f-1ebd0de8fb94 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:58.525815 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32c6e5-039b-4e6e-8e6f-1ebd0de8fb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:58.526091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32c6e5-039b-4e6e-8e6f-1ebd0de8fb94 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:58.533577 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32c6e5-039b-4e6e-8e6f-1ebd0de8fb94 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:58.535041 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32c6e5-039b-4e6e-8e6f-1ebd0de8fb94 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:58.546426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-ef32c6e5-039b-4e6e-8e6f-1ebd0de8fb94 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:08:58.546426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:08:58.546426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:08:58.546426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:08:58.546426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:08:58.546426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:08:58.546426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:08:58.546426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:08:58.546426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:08:58.546426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:08:58.546426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:08:58.546426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:08:58.546426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:08:58.546426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:08:58.546426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:08:58.546426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:08:58.548974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:08:58.550188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:08:58.550188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:08:58.550188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:08:58.550188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:08:58.550188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:08:58.550188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:08:58.550188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:08:58.550188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:08:58.550188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:08:58.550188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:08:58.550188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 604/1212] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:08:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:08:58.554100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d614b-21b8-4baf-995d-f9c17b6ba7a3 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:58.654376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d614b-21b8-4baf-995d-f9c17b6ba7a3 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:58.688991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f502bd0-1e20-4377-a2b0-08a8f84d49f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:58.689649 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f502bd0-1e20-4377-a2b0-08a8f84d49f4 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:58.689928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f502bd0-1e20-4377-a2b0-08a8f84d49f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:58.690202 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f502bd0-1e20-4377-a2b0-08a8f84d49f4 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:58.690959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f502bd0-1e20-4377-a2b0-08a8f84d49f4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:08:58.691270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f502bd0-1e20-4377-a2b0-08a8f84d49f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:58.692702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f502bd0-1e20-4377-a2b0-08a8f84d49f4 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:58.710652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 609/1213] 10.222.0.59 () {56 vars in 1143 bytes} [Wed Aug 31 19:08:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:08:58.716633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32c6e5-039b-4e6e-8e6f-1ebd0de8fb94 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:58.817788 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32c6e5-039b-4e6e-8e6f-1ebd0de8fb94 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:58.851867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e274b92-3319-489b-9e48-dd5aa4a8783a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:58.852476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e274b92-3319-489b-9e48-dd5aa4a8783a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:58.852759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e274b92-3319-489b-9e48-dd5aa4a8783a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:58.853034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e274b92-3319-489b-9e48-dd5aa4a8783a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:58.883559 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:58.921290 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e274b92-3319-489b-9e48-dd5aa4a8783a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:58.922776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e274b92-3319-489b-9e48-dd5aa4a8783a None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:59.023232 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 605/1214] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:08:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:08:59.029174 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f502bd0-1e20-4377-a2b0-08a8f84d49f4 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:59.048952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:59.130266 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f502bd0-1e20-4377-a2b0-08a8f84d49f4 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:59.165684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ac434-a5a6-49b7-a600-327809d47263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:59.166310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ac434-a5a6-49b7-a600-327809d47263 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:59.166587 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ac434-a5a6-49b7-a600-327809d47263 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:59.166856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ac434-a5a6-49b7-a600-327809d47263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:59.176831 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ac434-a5a6-49b7-a600-327809d47263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:59.178987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ac434-a5a6-49b7-a600-327809d47263 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:59.193658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-9a3ac434-a5a6-49b7-a600-327809d47263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:08:59.193658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:08:59.193658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:08:59.193658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:08:59.193658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:08:59.193658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:08:59.193658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:08:59.193658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:08:59.193658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:08:59.193658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:08:59.193658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:08:59.193658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:08:59.193658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:08:59.193658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:08:59.193658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:08:59.193658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:08:59.194938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:08:59.194938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:08:59.194938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:08:59.194938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:08:59.194938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:08:59.194938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:08:59.194938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:08:59.194938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:08:59.194938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:08:59.194938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:08:59.194938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:08:59.194938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:08:59.194938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:08:59.194938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:08:59.194938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:08:59.194938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:08:59.194938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:08:59.201955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:08:59.201955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:08:59.201955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:08:59.201955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 610/1215] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:08:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:08:59.202237 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e274b92-3319-489b-9e48-dd5aa4a8783a None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:59.264171 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:59.303255 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e274b92-3319-489b-9e48-dd5aa4a8783a None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:59.337660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30cfdd-3c59-46d4-aeca-12f9eb1f92f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:59.338239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30cfdd-3c59-46d4-aeca-12f9eb1f92f0 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:59.338520 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30cfdd-3c59-46d4-aeca-12f9eb1f92f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:59.338795 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30cfdd-3c59-46d4-aeca-12f9eb1f92f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:59.339464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30cfdd-3c59-46d4-aeca-12f9eb1f92f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:08:59.339773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30cfdd-3c59-46d4-aeca-12f9eb1f92f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:59.341131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30cfdd-3c59-46d4-aeca-12f9eb1f92f0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:59.352916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 606/1216] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:08:59.359045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ac434-a5a6-49b7-a600-327809d47263 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:59.429340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:59.460238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ac434-a5a6-49b7-a600-327809d47263 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:59.494116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd043f97-4279-4093-9bc7-a3629a14f87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:59.494748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd043f97-4279-4093-9bc7-a3629a14f87c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:59.495083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd043f97-4279-4093-9bc7-a3629a14f87c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:59.495361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd043f97-4279-4093-9bc7-a3629a14f87c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:59.502083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd043f97-4279-4093-9bc7-a3629a14f87c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:59.503676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd043f97-4279-4093-9bc7-a3629a14f87c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:59.515275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-fd043f97-4279-4093-9bc7-a3629a14f87c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:08:59.515275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:08:59.515275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:08:59.515275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:08:59.515275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:08:59.515275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:08:59.515275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:08:59.515275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:08:59.515275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:08:59.515275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:08:59.515275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:08:59.515275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:08:59.515275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:08:59.515275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:08:59.515275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:08:59.517658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:08:59.517658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:08:59.517658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:08:59.517658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:08:59.517658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:08:59.517658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:08:59.517658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:08:59.517658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:08:59.517658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:08:59.517658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:08:59.517658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:08:59.517658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:08:59.517658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:08:59.517658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:08:59.517658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:08:59.517658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:08:59.517658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:08:59.518588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:08:59.518588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:08:59.518588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:08:59.518588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:08:59.518588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:08:59.518588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:08:59.518588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:08:59.518588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:08:59.518588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 611/1217] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:08:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:08:59.522844 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30cfdd-3c59-46d4-aeca-12f9eb1f92f0 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:59.624565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30cfdd-3c59-46d4-aeca-12f9eb1f92f0 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:59.648171 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:59.659863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf250d-598a-4c0f-9c9a-dd3f353bbfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:59.660509 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf250d-598a-4c0f-9c9a-dd3f353bbfde None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:59.660791 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf250d-598a-4c0f-9c9a-dd3f353bbfde None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:59.661172 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf250d-598a-4c0f-9c9a-dd3f353bbfde None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:59.661863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf250d-598a-4c0f-9c9a-dd3f353bbfde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:08:59.662293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf250d-598a-4c0f-9c9a-dd3f353bbfde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:59.664033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf250d-598a-4c0f-9c9a-dd3f353bbfde None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:59.675822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 607/1218] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:08:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:08:59.686954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd043f97-4279-4093-9bc7-a3629a14f87c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:59.782484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd043f97-4279-4093-9bc7-a3629a14f87c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:59.810284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:59.816044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881ff15-7e6c-48ba-9ba2-dcfcb4e8e2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:59.816655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881ff15-7e6c-48ba-9ba2-dcfcb4e8e2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:59.816941 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881ff15-7e6c-48ba-9ba2-dcfcb4e8e2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:59.817216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881ff15-7e6c-48ba-9ba2-dcfcb4e8e2c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:59.817947 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881ff15-7e6c-48ba-9ba2-dcfcb4e8e2c8 None admin] RBAC: Adding query filter params (role.id=a9ef0098e982437fb0127827fa2cad38, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=eced13cc4030423abe63bb1527f6d35a) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:08:59.818260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881ff15-7e6c-48ba-9ba2-dcfcb4e8e2c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:59.820414 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881ff15-7e6c-48ba-9ba2-dcfcb4e8e2c8 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:59.825990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 612/1219] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:08:59 2022] GET /identity/v3/role_assignments?user.id=eced13cc4030423abe63bb1527f6d35a&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=a9ef0098e982437fb0127827fa2cad38 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:09:01.753422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da337b-7c4a-488d-9bfc-8e9cf642f58d None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:01.754044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da337b-7c4a-488d-9bfc-8e9cf642f58d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:01.754161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da337b-7c4a-488d-9bfc-8e9cf642f58d None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:01.755509 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 608/1220] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:01.761112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03488dfc-d3f4-43fd-8bb2-0d92416f6397 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:01.761571 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03488dfc-d3f4-43fd-8bb2-0d92416f6397 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:01.761667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03488dfc-d3f4-43fd-8bb2-0d92416f6397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:01.780285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-03488dfc-d3f4-43fd-8bb2-0d92416f6397 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:01.781539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-03488dfc-d3f4-43fd-8bb2-0d92416f6397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:01.850154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 613/1221] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:09:02.124301 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da337b-7c4a-488d-9bfc-8e9cf642f58d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:02.227254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da337b-7c4a-488d-9bfc-8e9cf642f58d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:02.261618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589dd16-924f-475c-8ae0-d2d77eaf5806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:02.262256 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589dd16-924f-475c-8ae0-d2d77eaf5806 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:02.262515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589dd16-924f-475c-8ae0-d2d77eaf5806 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:02.262782 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589dd16-924f-475c-8ae0-d2d77eaf5806 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:02.274848 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0589dd16-924f-475c-8ae0-d2d77eaf5806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:09:02.279703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-0589dd16-924f-475c-8ae0-d2d77eaf5806 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:09:02.279703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:02.279703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:02.279703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:02.279703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:02.279703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:02.279703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:02.279703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:02.279703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:02.279703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:02.279703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:02.279703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:02.279703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:09:02.279703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:09:02.279703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:09:02.279703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:09:02.280852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:09:02.280852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:09:02.280852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:09:02.280852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:09:02.280852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:02.280852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:02.280852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:02.280852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:02.280852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:02.280852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:02.280852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:02.280852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:02.280852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:02.280852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:02.280852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:02.280852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:02.281767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:02.281767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:02.281767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:02.281767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:02.281767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:09:02.281767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:09:02.281767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:09:02.281767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:09:02.281767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:09:02.281767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:09:02.281767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:09:02.281767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:02.281767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 609/1222] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:09:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:02.287229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03488dfc-d3f4-43fd-8bb2-0d92416f6397 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:02.388287 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03488dfc-d3f4-43fd-8bb2-0d92416f6397 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:02.412225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:02.422631 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bbe7f-14be-444e-bce2-e7ab0315692b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:02.423269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bbe7f-14be-444e-bce2-e7ab0315692b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:02.423548 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bbe7f-14be-444e-bce2-e7ab0315692b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:02.423817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bbe7f-14be-444e-bce2-e7ab0315692b None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:02.424473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790bbe7f-14be-444e-bce2-e7ab0315692b None admin] RBAC: Adding query filter params (name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:02.424772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790bbe7f-14be-444e-bce2-e7ab0315692b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:02.426120 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790bbe7f-14be-444e-bce2-e7ab0315692b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:02.434446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 614/1223] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:09:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:09:02.440341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589dd16-924f-475c-8ae0-d2d77eaf5806 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:02.485508 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:02.542427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589dd16-924f-475c-8ae0-d2d77eaf5806 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:02.576530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d4d14-21a1-480d-9747-b9753df011bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:02.577156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d4d14-21a1-480d-9747-b9753df011bc None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:02.577432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d4d14-21a1-480d-9747-b9753df011bc None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:02.577702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d4d14-21a1-480d-9747-b9753df011bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:02.588783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d4d14-21a1-480d-9747-b9753df011bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:02.590218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d4d14-21a1-480d-9747-b9753df011bc None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:02.605528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-dc0d4d14-21a1-480d-9747-b9753df011bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:02.605528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:02.605528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:02.605528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:02.605528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:02.605528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:02.605528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:02.605528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:02.605528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:02.605528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:02.605528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:02.605528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:02.605528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:09:02.605528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:09:02.605528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:09:02.605528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:09:02.607263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:02.607263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:02.607263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:02.607263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:02.607263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:02.607263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:02.607263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:02.607263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:02.607263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:02.607263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:02.607263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:02.607263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:02.607263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:02.607263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:02.607263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:02.607263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:02.607263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:09:02.608997 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:09:02.608997 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:02.608997 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:02.608997 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 610/1224] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:02.612749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bbe7f-14be-444e-bce2-e7ab0315692b None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:02.712947 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bbe7f-14be-444e-bce2-e7ab0315692b None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:02.747502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa703d-7c70-4d51-80eb-7dc8c9e90a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:02.748109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa703d-7c70-4d51-80eb-7dc8c9e90a3c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:02.748390 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa703d-7c70-4d51-80eb-7dc8c9e90a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:02.748662 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa703d-7c70-4d51-80eb-7dc8c9e90a3c None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:02.749303 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa703d-7c70-4d51-80eb-7dc8c9e90a3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:02.749607 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa703d-7c70-4d51-80eb-7dc8c9e90a3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:02.751106 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa703d-7c70-4d51-80eb-7dc8c9e90a3c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:02.762665 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 615/1225] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:02.769248 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d4d14-21a1-480d-9747-b9753df011bc None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:02.870813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:02.877054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d4d14-21a1-480d-9747-b9753df011bc None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:02.909287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae683168-1dba-4a60-91a5-6620742202a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:02.909911 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae683168-1dba-4a60-91a5-6620742202a7 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:02.910186 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae683168-1dba-4a60-91a5-6620742202a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:02.910454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae683168-1dba-4a60-91a5-6620742202a7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:02.918459 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae683168-1dba-4a60-91a5-6620742202a7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:02.920545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae683168-1dba-4a60-91a5-6620742202a7 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:02.932984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-ae683168-1dba-4a60-91a5-6620742202a7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:09:02.932984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:02.932984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:02.932984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:02.932984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:02.932984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:02.932984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:02.932984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:02.932984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:02.932984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:02.932984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:02.932984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:02.932984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:09:02.932984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:09:02.932984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:09:02.932984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:02.934245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:02.935846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:02.935846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:02.935846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:09:02.935846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:09:02.935846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:09:02.935846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:09:02.935846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:09:02.935846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:09:02.935846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:09:02.935846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:02.935846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 611/1226] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:09:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:02.940038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa703d-7c70-4d51-80eb-7dc8c9e90a3c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:02.987541 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:03.039833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa703d-7c70-4d51-80eb-7dc8c9e90a3c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:03.075084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00451a23-2eac-4c95-9d85-dc1e08143467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:03.075690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00451a23-2eac-4c95-9d85-dc1e08143467 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:03.075967 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00451a23-2eac-4c95-9d85-dc1e08143467 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:03.076234 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00451a23-2eac-4c95-9d85-dc1e08143467 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:03.077028 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00451a23-2eac-4c95-9d85-dc1e08143467 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:03.077331 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00451a23-2eac-4c95-9d85-dc1e08143467 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:03.078747 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00451a23-2eac-4c95-9d85-dc1e08143467 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:03.096078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 616/1227] 10.222.0.59 () {56 vars in 1143 bytes} [Wed Aug 31 19:09:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:03.102919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae683168-1dba-4a60-91a5-6620742202a7 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:03.206541 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae683168-1dba-4a60-91a5-6620742202a7 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:03.240792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ccaa83-58ca-4bf0-b27a-53c3fc0ca3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:03.241614 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ccaa83-58ca-4bf0-b27a-53c3fc0ca3ef None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:03.241920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ccaa83-58ca-4bf0-b27a-53c3fc0ca3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:03.242195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ccaa83-58ca-4bf0-b27a-53c3fc0ca3ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:03.252542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ccaa83-58ca-4bf0-b27a-53c3fc0ca3ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:03.253980 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ccaa83-58ca-4bf0-b27a-53c3fc0ca3ef None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:03.268603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-75ccaa83-58ca-4bf0-b27a-53c3fc0ca3ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:03.268603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:03.268603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:03.268603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:03.268603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:03.268603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:03.268603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:03.268603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:03.268603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:03.268603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:03.268603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:03.268603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:03.268603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:09:03.268603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:09:03.268603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:09:03.268603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:09:03.273648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:03.273648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:03.273648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:03.273648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:03.273648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:03.273648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:03.273648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:03.273648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:03.273648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:03.273648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:03.273648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:03.273648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:03.273648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:03.273648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:03.273648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:03.273648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:03.273648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:09:03.278796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:09:03.278796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:03.278796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:03.278796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 612/1228] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:09:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:03.279154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00451a23-2eac-4c95-9d85-dc1e08143467 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:03.377107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00451a23-2eac-4c95-9d85-dc1e08143467 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:03.379407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:03.419458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dc6d4-3bd4-484b-b164-e47487d51d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:03.419895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dc6d4-3bd4-484b-b164-e47487d51d95 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:03.420175 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dc6d4-3bd4-484b-b164-e47487d51d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:03.420450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dc6d4-3bd4-484b-b164-e47487d51d95 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:03.421099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9dc6d4-3bd4-484b-b164-e47487d51d95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:03.421407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9dc6d4-3bd4-484b-b164-e47487d51d95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:03.422787 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9dc6d4-3bd4-484b-b164-e47487d51d95 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:03.436191 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 617/1229] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:09:03.441992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ccaa83-58ca-4bf0-b27a-53c3fc0ca3ef None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:03.452290 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:03.563507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ccaa83-58ca-4bf0-b27a-53c3fc0ca3ef None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:03.601201 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5895af-8a9c-47e4-8c29-023874b5b4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:03.601895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5895af-8a9c-47e4-8c29-023874b5b4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:03.602176 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5895af-8a9c-47e4-8c29-023874b5b4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:03.602449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5895af-8a9c-47e4-8c29-023874b5b4d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:03.610149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5895af-8a9c-47e4-8c29-023874b5b4d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:03.611630 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5895af-8a9c-47e4-8c29-023874b5b4d0 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:03.623329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-bd5895af-8a9c-47e4-8c29-023874b5b4d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:09:03.623329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:03.623329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:03.623329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:03.623329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:03.623329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:03.623329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:03.623329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:03.623329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:03.623329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:03.623329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:03.623329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:03.623329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:09:03.623329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:09:03.623329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:09:03.624383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:09:03.624383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:03.624383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:03.624383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:03.624383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:03.624383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:03.624383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:03.624383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:03.624383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:03.624383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:03.624383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:03.624383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:03.624383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:03.624383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:03.624383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:03.624383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:03.624383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:03.625492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:09:03.625492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:09:03.625492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:09:03.625492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:09:03.625492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:09:03.625492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:09:03.625492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:09:03.625492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:03.625492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 613/1230] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:09:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:03.630306 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dc6d4-3bd4-484b-b164-e47487d51d95 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:03.732187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dc6d4-3bd4-484b-b164-e47487d51d95 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:03.766553 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee49a2-a130-4658-af6d-628143d5a6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:03.767242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee49a2-a130-4658-af6d-628143d5a6ec None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:03.767458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee49a2-a130-4658-af6d-628143d5a6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:03.767728 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee49a2-a130-4658-af6d-628143d5a6ec None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:03.768416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee49a2-a130-4658-af6d-628143d5a6ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:03.768725 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee49a2-a130-4658-af6d-628143d5a6ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:03.770146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee49a2-a130-4658-af6d-628143d5a6ec None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:03.783213 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 618/1231] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:09:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:03.789466 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5895af-8a9c-47e4-8c29-023874b5b4d0 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:03.853574 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:03.888719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5895af-8a9c-47e4-8c29-023874b5b4d0 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:03.922377 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49498524-0e77-49f2-afca-f2d2118825e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:03.923131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49498524-0e77-49f2-afca-f2d2118825e9 None admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:03.923414 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49498524-0e77-49f2-afca-f2d2118825e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:03.923691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49498524-0e77-49f2-afca-f2d2118825e9 None admin] PATH_INFO: `/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/eced13cc4030423abe63bb1527f6d35a/roles/a9ef0098e982437fb0127827fa2cad38` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:03.938444 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49498524-0e77-49f2-afca-f2d2118825e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=161c90d34bb34925a1aaeb70c7de7eae, user_id=eced13cc4030423abe63bb1527f6d35a, role_id=a9ef0098e982437fb0127827fa2cad38)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:03.939915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49498524-0e77-49f2-afca-f2d2118825e9 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:04.085860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 614/1232] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:09:03 2022] PUT /identity/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/eced13cc4030423abe63bb1527f6d35a/roles/a9ef0098e982437fb0127827fa2cad38 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:09:05.984298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bd3a7-dca1-4727-8375-4d8783aabc21 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:05.984861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bd3a7-dca1-4727-8375-4d8783aabc21 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:05.984861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bd3a7-dca1-4727-8375-4d8783aabc21 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:05.986172 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 619/1233] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:05.991441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae77732-c7dd-4c3a-86d3-0c6a24630274 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:05.991849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae77732-c7dd-4c3a-86d3-0c6a24630274 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:05.992023 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae77732-c7dd-4c3a-86d3-0c6a24630274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:06.009962 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-aae77732-c7dd-4c3a-86d3-0c6a24630274 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:06.014439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-aae77732-c7dd-4c3a-86d3-0c6a24630274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:06.082428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 615/1234] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:09:06.354529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bd3a7-dca1-4727-8375-4d8783aabc21 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:06.381234 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:06.458505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bd3a7-dca1-4727-8375-4d8783aabc21 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:06.493867 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465a6f7-43b4-4c9e-aac8-a1db6efbd457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:06.494502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465a6f7-43b4-4c9e-aac8-a1db6efbd457 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:06.494781 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465a6f7-43b4-4c9e-aac8-a1db6efbd457 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:06.495081 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465a6f7-43b4-4c9e-aac8-a1db6efbd457 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:06.505482 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465a6f7-43b4-4c9e-aac8-a1db6efbd457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:09:06.510812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-8465a6f7-43b4-4c9e-aac8-a1db6efbd457 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:09:06.510812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:06.510812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:06.510812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:06.510812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:06.510812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:06.510812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:06.510812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:06.510812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:06.510812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:06.510812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:06.510812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:06.510812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:09:06.510812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:09:06.510812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:09:06.510812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:09:06.512253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:09:06.512253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:09:06.512253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:09:06.512253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:09:06.512253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:06.512253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:06.512253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:06.512253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:06.512253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:06.512253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:06.512253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:06.512253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:06.512253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:06.512253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:06.512253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:06.512253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:06.513157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:06.513157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:06.513157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:06.513157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:06.513157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:09:06.513157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:09:06.513157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:09:06.513157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:09:06.513157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:09:06.513157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:09:06.513157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:09:06.513157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:09:06.513157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 620/1235] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 31 19:09:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:06.518078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae77732-c7dd-4c3a-86d3-0c6a24630274 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:06.565589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:06.618375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae77732-c7dd-4c3a-86d3-0c6a24630274 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:06.653394 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b9725-24c4-4065-8a78-75444810c141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:06.654068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b9725-24c4-4065-8a78-75444810c141 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:06.654418 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b9725-24c4-4065-8a78-75444810c141 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:06.654696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b9725-24c4-4065-8a78-75444810c141 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:06.655530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b9725-24c4-4065-8a78-75444810c141 None admin] RBAC: Adding query filter params (name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:06.655839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b9725-24c4-4065-8a78-75444810c141 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:06.657199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b9725-24c4-4065-8a78-75444810c141 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:06.665816 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 616/1236] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 31 19:09:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:06.671662 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465a6f7-43b4-4c9e-aac8-a1db6efbd457 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:06.756806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:06.772941 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465a6f7-43b4-4c9e-aac8-a1db6efbd457 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:06.806682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa6ed5-34c5-4d63-8681-a57b2719b452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:06.807354 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa6ed5-34c5-4d63-8681-a57b2719b452 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:06.807636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa6ed5-34c5-4d63-8681-a57b2719b452 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:06.807912 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa6ed5-34c5-4d63-8681-a57b2719b452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:06.818100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa6ed5-34c5-4d63-8681-a57b2719b452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:06.819719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa6ed5-34c5-4d63-8681-a57b2719b452 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:06.833585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-bbfa6ed5-34c5-4d63-8681-a57b2719b452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:06.833585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:06.833585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:06.833585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:06.833585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:06.833585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:06.833585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:06.833585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:06.833585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:06.833585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:06.833585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:06.833585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:06.833585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:09:06.833585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:09:06.833585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:09:06.833585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:09:06.835230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:06.835230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:06.835230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:06.835230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:06.835230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:06.835230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:06.835230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:06.835230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:06.835230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:06.835230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:06.835230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:06.835230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:06.835230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:06.835230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:06.835230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:06.835230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:06.835230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:09:06.836154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:09:06.836154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:06.836154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:09:06.836154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 621/1237] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:06.841436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b9725-24c4-4065-8a78-75444810c141 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:06.944464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b9725-24c4-4065-8a78-75444810c141 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:06.953361 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:06.979465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffde393-a406-4453-af2c-4d3adb394cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:06.980075 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffde393-a406-4453-af2c-4d3adb394cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:06.980359 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffde393-a406-4453-af2c-4d3adb394cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:06.980668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffde393-a406-4453-af2c-4d3adb394cf3 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:06.981364 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffde393-a406-4453-af2c-4d3adb394cf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:06.981672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffde393-a406-4453-af2c-4d3adb394cf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:06.983408 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffde393-a406-4453-af2c-4d3adb394cf3 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:06.995368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 617/1238] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:07.001322 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa6ed5-34c5-4d63-8681-a57b2719b452 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:07.101496 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa6ed5-34c5-4d63-8681-a57b2719b452 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:07.136606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc457e1-e010-495a-8e1b-639da6df7d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:07.137263 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc457e1-e010-495a-8e1b-639da6df7d61 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:07.137540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc457e1-e010-495a-8e1b-639da6df7d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:07.137811 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc457e1-e010-495a-8e1b-639da6df7d61 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:07.140475 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:07.145509 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc457e1-e010-495a-8e1b-639da6df7d61 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:07.148035 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc457e1-e010-495a-8e1b-639da6df7d61 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:07.159569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-6dc457e1-e010-495a-8e1b-639da6df7d61 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:09:07.159569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:07.159569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:07.159569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:07.159569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:07.159569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:07.159569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:07.159569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:07.159569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:07.159569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:07.159569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:07.159569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:07.159569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:09:07.159569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:09:07.159569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:09:07.159569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:07.160642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:07.161944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:07.161944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:07.161944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:09:07.161944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:09:07.161944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:09:07.161944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:09:07.161944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:09:07.161944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:09:07.161944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:09:07.161944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:09:07.161944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 622/1239] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:09:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:07.166938 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffde393-a406-4453-af2c-4d3adb394cf3 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:07.266918 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffde393-a406-4453-af2c-4d3adb394cf3 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:07.302833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad20e49-cdcb-4859-9b79-1c698c416521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:07.303477 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad20e49-cdcb-4859-9b79-1c698c416521 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:07.303987 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad20e49-cdcb-4859-9b79-1c698c416521 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:07.304178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad20e49-cdcb-4859-9b79-1c698c416521 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:07.304903 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad20e49-cdcb-4859-9b79-1c698c416521 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:07.305212 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad20e49-cdcb-4859-9b79-1c698c416521 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:07.306810 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad20e49-cdcb-4859-9b79-1c698c416521 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:07.324859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 618/1240] 10.222.0.59 () {56 vars in 1143 bytes} [Wed Aug 31 19:09:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:07.330781 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc457e1-e010-495a-8e1b-639da6df7d61 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:07.431369 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc457e1-e010-495a-8e1b-639da6df7d61 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:07.467222 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67e44c-f429-4a70-95b2-acd914e5dc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:07.467829 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67e44c-f429-4a70-95b2-acd914e5dc68 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:07.468114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67e44c-f429-4a70-95b2-acd914e5dc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:07.468394 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67e44c-f429-4a70-95b2-acd914e5dc68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:07.534617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:07.541402 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67e44c-f429-4a70-95b2-acd914e5dc68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:07.542856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67e44c-f429-4a70-95b2-acd914e5dc68 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:07.641339 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 623/1241] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:09:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:09:07.647475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad20e49-cdcb-4859-9b79-1c698c416521 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:07.674743 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:07.748516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad20e49-cdcb-4859-9b79-1c698c416521 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:07.781851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443eba5-f6e9-4eb1-a288-ed44e6eb14a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:07.783585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443eba5-f6e9-4eb1-a288-ed44e6eb14a9 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:07.783585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443eba5-f6e9-4eb1-a288-ed44e6eb14a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:07.783585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443eba5-f6e9-4eb1-a288-ed44e6eb14a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:07.793678 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443eba5-f6e9-4eb1-a288-ed44e6eb14a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:07.795128 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443eba5-f6e9-4eb1-a288-ed44e6eb14a9 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:07.811169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-5443eba5-f6e9-4eb1-a288-ed44e6eb14a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:07.811169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:07.811169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:07.811169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:07.811169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:07.811169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:07.811169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:07.811169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:07.811169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:07.811169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:07.811169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:07.811169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:07.811169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:09:07.811169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:09:07.811169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:09:07.811169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:09:07.818835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:07.818835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:07.818835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:07.818835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:07.818835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:07.818835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:07.818835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:07.818835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:07.818835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:07.818835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:07.818835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:07.818835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:07.818835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:07.818835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:07.818835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:07.818835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:07.818835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:09:07.819863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:09:07.819863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:07.819863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:07.819863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 619/1242] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:07.820108 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67e44c-f429-4a70-95b2-acd914e5dc68 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:07.909020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:07.915473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67e44c-f429-4a70-95b2-acd914e5dc68 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:07.959996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14030d-e753-49da-be35-994f3fc8056c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:07.960612 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14030d-e753-49da-be35-994f3fc8056c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:07.960893 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14030d-e753-49da-be35-994f3fc8056c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:07.961170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14030d-e753-49da-be35-994f3fc8056c None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:07.961814 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14030d-e753-49da-be35-994f3fc8056c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:07.962120 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14030d-e753-49da-be35-994f3fc8056c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:07.963617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14030d-e753-49da-be35-994f3fc8056c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:07.975799 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 624/1243] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:07.981434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443eba5-f6e9-4eb1-a288-ed44e6eb14a9 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:08.064307 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:08.086731 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443eba5-f6e9-4eb1-a288-ed44e6eb14a9 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:08.121979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afcaef-ee69-46b4-8cd7-2e904c363998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:08.122700 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afcaef-ee69-46b4-8cd7-2e904c363998 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:08.123108 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afcaef-ee69-46b4-8cd7-2e904c363998 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:08.123434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afcaef-ee69-46b4-8cd7-2e904c363998 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:08.130266 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91afcaef-ee69-46b4-8cd7-2e904c363998 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:08.131888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91afcaef-ee69-46b4-8cd7-2e904c363998 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:08.143452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-91afcaef-ee69-46b4-8cd7-2e904c363998 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:09:08.143452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:08.143452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:08.143452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:08.143452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:08.143452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:08.143452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:08.143452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:08.143452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:08.143452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:08.143452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:08.143452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:08.143452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:09:08.143452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:09:08.143452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:09:08.144576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:09:08.144576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:08.144576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:08.144576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:08.144576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:08.144576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:08.144576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:08.144576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:08.144576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:08.144576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:08.144576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:08.144576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:08.144576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:08.144576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:08.144576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:08.144576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:08.144576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:08.145505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:09:08.145505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:09:08.145505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:09:08.145505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:09:08.145505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:09:08.145505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:09:08.145505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:09:08.145505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:08.145505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 620/1244] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:09:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:09:08.150752 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14030d-e753-49da-be35-994f3fc8056c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:08.254001 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14030d-e753-49da-be35-994f3fc8056c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:08.294004 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451e102-f7af-4ca4-bd4d-0c1fd55dbbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:08.294615 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451e102-f7af-4ca4-bd4d-0c1fd55dbbf0 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:08.294920 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451e102-f7af-4ca4-bd4d-0c1fd55dbbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:08.295194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451e102-f7af-4ca4-bd4d-0c1fd55dbbf0 None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:08.295884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451e102-f7af-4ca4-bd4d-0c1fd55dbbf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:08.296185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451e102-f7af-4ca4-bd4d-0c1fd55dbbf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:08.297603 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451e102-f7af-4ca4-bd4d-0c1fd55dbbf0 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:08.309404 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 625/1245] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:09:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:08.315704 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afcaef-ee69-46b4-8cd7-2e904c363998 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:08.417029 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afcaef-ee69-46b4-8cd7-2e904c363998 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:08.449351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81766586-281d-42cb-a865-fd522a24847e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:08.449954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81766586-281d-42cb-a865-fd522a24847e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:08.450228 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81766586-281d-42cb-a865-fd522a24847e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:08.450496 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81766586-281d-42cb-a865-fd522a24847e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:08.451234 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81766586-281d-42cb-a865-fd522a24847e None admin] RBAC: Adding query filter params (role.id=a9ef0098e982437fb0127827fa2cad38, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=eced13cc4030423abe63bb1527f6d35a) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:08.451538 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81766586-281d-42cb-a865-fd522a24847e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:08.452960 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81766586-281d-42cb-a865-fd522a24847e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:08.459686 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 621/1246] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:09:08 2022] GET /identity/v3/role_assignments?user.id=eced13cc4030423abe63bb1527f6d35a&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=a9ef0098e982437fb0127827fa2cad38 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:10.434813 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db2b6e-1c0f-4ac0-a522-faff08cf036e None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:10.435365 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db2b6e-1c0f-4ac0-a522-faff08cf036e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:10.435461 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db2b6e-1c0f-4ac0-a522-faff08cf036e None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:10.436735 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 626/1247] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:10.441712 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbb93c-e60a-445d-8263-039a25f21035 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:10.442164 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbb93c-e60a-445d-8263-039a25f21035 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:10.442299 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbb93c-e60a-445d-8263-039a25f21035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:10.449462 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:10.461624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-eafbb93c-e60a-445d-8263-039a25f21035 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:10.462812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-eafbb93c-e60a-445d-8263-039a25f21035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:10.532025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 622/1248] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:09:10.812308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db2b6e-1c0f-4ac0-a522-faff08cf036e None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:10.830805 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:10.916354 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db2b6e-1c0f-4ac0-a522-faff08cf036e None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:10.950494 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e93f92-6ae4-4fb8-8809-0400dae7d19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:10.951155 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e93f92-6ae4-4fb8-8809-0400dae7d19f None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:10.951439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e93f92-6ae4-4fb8-8809-0400dae7d19f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:10.951717 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e93f92-6ae4-4fb8-8809-0400dae7d19f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:10.962174 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e93f92-6ae4-4fb8-8809-0400dae7d19f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:09:10.967161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-a6e93f92-6ae4-4fb8-8809-0400dae7d19f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:09:10.967161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:10.967161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:10.967161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:10.967161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:10.967161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:10.967161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:10.967161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:10.967161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:10.967161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:10.967161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:10.967161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:10.967161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:09:10.967161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:09:10.967161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:09:10.967161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:09:10.968276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:09:10.968276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:09:10.968276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:09:10.968276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:09:10.968276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:10.968276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:10.968276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:10.968276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:10.968276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:10.968276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:10.968276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:10.968276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:10.968276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:10.968276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:10.968276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:10.968276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:10.969329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:10.969329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:10.969329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:10.969329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:10.969329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:09:10.969329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:09:10.969329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:09:10.969329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:09:10.969329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:09:10.969329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:09:10.969329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:09:10.969329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:09:10.969329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 627/1249] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:09:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:10.975051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbb93c-e60a-445d-8263-039a25f21035 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:11.076275 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbb93c-e60a-445d-8263-039a25f21035 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:11.105321 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:11.110381 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e90f9-f5ba-49d4-98ea-4344d6f6b7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:11.111017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e90f9-f5ba-49d4-98ea-4344d6f6b7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:11.111297 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e90f9-f5ba-49d4-98ea-4344d6f6b7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:11.111566 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e90f9-f5ba-49d4-98ea-4344d6f6b7b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:11.112241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e90f9-f5ba-49d4-98ea-4344d6f6b7b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:11.112544 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e90f9-f5ba-49d4-98ea-4344d6f6b7b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:11.113889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e90f9-f5ba-49d4-98ea-4344d6f6b7b2 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:11.123016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 623/1250] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:09:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:11.128860 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e93f92-6ae4-4fb8-8809-0400dae7d19f None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:11.207626 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:11.231582 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e93f92-6ae4-4fb8-8809-0400dae7d19f None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:11.266606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb9725-099f-43e9-8b71-ea4aa5368a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:11.267273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb9725-099f-43e9-8b71-ea4aa5368a4b None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:11.267555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb9725-099f-43e9-8b71-ea4aa5368a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:11.267831 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb9725-099f-43e9-8b71-ea4aa5368a4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:11.278773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb9725-099f-43e9-8b71-ea4aa5368a4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:11.280378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb9725-099f-43e9-8b71-ea4aa5368a4b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:11.296527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-6dfb9725-099f-43e9-8b71-ea4aa5368a4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:11.296527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:11.296527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:11.296527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:11.296527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:11.296527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:11.296527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:11.296527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:11.296527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:11.296527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:11.296527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:11.296527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:11.296527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:09:11.296527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:09:11.296527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:09:11.296527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:09:11.297719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:11.297719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:11.297719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:11.297719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:11.297719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:11.297719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:11.297719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:11.297719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:11.297719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:11.297719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:11.297719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:11.297719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:11.297719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:11.297719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:11.297719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:11.297719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:11.297719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:09:11.298921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:09:11.298921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:11.298921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:09:11.298921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 628/1251] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:11.303547 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e90f9-f5ba-49d4-98ea-4344d6f6b7b2 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:11.406672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e90f9-f5ba-49d4-98ea-4344d6f6b7b2 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:11.440956 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7840e8-9c24-4db2-848f-43db2e593f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:11.441563 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7840e8-9c24-4db2-848f-43db2e593f35 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:11.441846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7840e8-9c24-4db2-848f-43db2e593f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:11.442443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7840e8-9c24-4db2-848f-43db2e593f35 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:11.442749 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7840e8-9c24-4db2-848f-43db2e593f35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:11.443085 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7840e8-9c24-4db2-848f-43db2e593f35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:11.444524 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7840e8-9c24-4db2-848f-43db2e593f35 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:11.456220 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 624/1252] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:11.462678 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb9725-099f-43e9-8b71-ea4aa5368a4b None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:11.564389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb9725-099f-43e9-8b71-ea4aa5368a4b None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:11.590625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:11.598406 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12919c2-70d9-4ea2-a7ab-b243147f2616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:11.599066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12919c2-70d9-4ea2-a7ab-b243147f2616 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:11.599344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12919c2-70d9-4ea2-a7ab-b243147f2616 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:11.599881 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12919c2-70d9-4ea2-a7ab-b243147f2616 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:11.607170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12919c2-70d9-4ea2-a7ab-b243147f2616 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:11.608849 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12919c2-70d9-4ea2-a7ab-b243147f2616 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:11.619667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-c12919c2-70d9-4ea2-a7ab-b243147f2616 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:09:11.619667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:11.619667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:11.619667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:11.619667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:11.619667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:11.619667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:11.619667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:11.619667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:11.619667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:11.619667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:11.619667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:11.619667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:09:11.619667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:09:11.619667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:09:11.619667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:11.623684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:11.624929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:11.624929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:11.624929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:09:11.624929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:09:11.624929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:09:11.624929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:09:11.624929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:09:11.624929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:09:11.624929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:09:11.624929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:09:11.624929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 629/1253] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:09:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:11.626737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7840e8-9c24-4db2-848f-43db2e593f35 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:11.684569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:11.729135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7840e8-9c24-4db2-848f-43db2e593f35 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:11.764870 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8246da3-4517-4c50-a9b3-b0a7be63c5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:11.765484 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8246da3-4517-4c50-a9b3-b0a7be63c5d7 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:11.765762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8246da3-4517-4c50-a9b3-b0a7be63c5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:11.766035 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8246da3-4517-4c50-a9b3-b0a7be63c5d7 None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:11.766756 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8246da3-4517-4c50-a9b3-b0a7be63c5d7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:11.767088 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8246da3-4517-4c50-a9b3-b0a7be63c5d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:11.768505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8246da3-4517-4c50-a9b3-b0a7be63c5d7 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:11.785974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 625/1254] 10.222.0.59 () {56 vars in 1143 bytes} [Wed Aug 31 19:09:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:11.792135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12919c2-70d9-4ea2-a7ab-b243147f2616 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:11.893379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12919c2-70d9-4ea2-a7ab-b243147f2616 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:11.927072 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f6892-15f1-4eac-9b25-cd4503bd8520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:11.928247 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f6892-15f1-4eac-9b25-cd4503bd8520 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:11.928440 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f6892-15f1-4eac-9b25-cd4503bd8520 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:11.928708 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f6892-15f1-4eac-9b25-cd4503bd8520 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:11.982634 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:11.995091 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f6892-15f1-4eac-9b25-cd4503bd8520 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:11.996531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f6892-15f1-4eac-9b25-cd4503bd8520 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:12.094123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 630/1255] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:09:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:09:12.100522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8246da3-4517-4c50-a9b3-b0a7be63c5d7 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:12.203970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8246da3-4517-4c50-a9b3-b0a7be63c5d7 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:12.216983 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:12.238298 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15886474-b75c-4086-806a-aebeb2485a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:12.238862 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15886474-b75c-4086-806a-aebeb2485a6b None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:12.239166 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15886474-b75c-4086-806a-aebeb2485a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:12.239439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15886474-b75c-4086-806a-aebeb2485a6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:12.250366 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15886474-b75c-4086-806a-aebeb2485a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:12.251821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15886474-b75c-4086-806a-aebeb2485a6b None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:12.266745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-15886474-b75c-4086-806a-aebeb2485a6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:12.266745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:12.266745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:12.266745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:12.266745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:12.266745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:12.266745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:12.266745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:12.266745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:12.266745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:12.266745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:12.266745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:12.266745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:09:12.266745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:09:12.266745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:09:12.266745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:09:12.267964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:12.267964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:12.267964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:12.267964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:12.267964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:12.267964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:12.267964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:12.267964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:12.267964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:12.267964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:12.267964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:12.267964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:12.267964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:12.267964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:12.267964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:12.267964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:12.267964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:09:12.269123 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:09:12.269123 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:12.269123 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:12.269123 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 626/1256] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:12.273945 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f6892-15f1-4eac-9b25-cd4503bd8520 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:12.361644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:12.374191 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f6892-15f1-4eac-9b25-cd4503bd8520 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:12.407892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39160b4-138d-4aa9-89f2-cc1c9d4d81f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:12.408530 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39160b4-138d-4aa9-89f2-cc1c9d4d81f2 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:12.408904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39160b4-138d-4aa9-89f2-cc1c9d4d81f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:12.409179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39160b4-138d-4aa9-89f2-cc1c9d4d81f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:12.409827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39160b4-138d-4aa9-89f2-cc1c9d4d81f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:12.410131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39160b4-138d-4aa9-89f2-cc1c9d4d81f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:12.411540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39160b4-138d-4aa9-89f2-cc1c9d4d81f2 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:12.423460 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 631/1257] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:09:12.429834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15886474-b75c-4086-806a-aebeb2485a6b None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:12.531945 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15886474-b75c-4086-806a-aebeb2485a6b None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:12.564616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a18d28-889b-4246-b10e-581f01d7cdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:12.565232 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a18d28-889b-4246-b10e-581f01d7cdbe None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:12.565508 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a18d28-889b-4246-b10e-581f01d7cdbe None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:12.565780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a18d28-889b-4246-b10e-581f01d7cdbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:12.572970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a18d28-889b-4246-b10e-581f01d7cdbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:12.574401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a18d28-889b-4246-b10e-581f01d7cdbe None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:12.586435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-a1a18d28-889b-4246-b10e-581f01d7cdbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:09:12.586435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:12.586435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:12.586435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:12.586435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:12.586435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:12.586435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:12.586435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:12.586435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:12.586435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:12.586435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:12.586435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:12.586435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:09:12.586435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:09:12.586435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:09:12.587550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:09:12.587550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:12.587550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:12.587550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:12.587550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:12.587550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:12.587550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:12.587550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:12.587550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:12.587550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:12.587550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:12.587550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:12.587550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:12.587550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:12.587550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:12.587550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:12.587550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:12.588631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:09:12.588631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:09:12.588631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:09:12.588631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:09:12.588631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:09:12.588631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:09:12.588631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:09:12.588631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:12.588631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 627/1258] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:09:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:09:12.593706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39160b4-138d-4aa9-89f2-cc1c9d4d81f2 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:12.699117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39160b4-138d-4aa9-89f2-cc1c9d4d81f2 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:12.732649 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21f0e5-89a5-4236-b25d-78dd4390d228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:12.733258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21f0e5-89a5-4236-b25d-78dd4390d228 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:12.733539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21f0e5-89a5-4236-b25d-78dd4390d228 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:12.733813 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21f0e5-89a5-4236-b25d-78dd4390d228 None admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:12.734503 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21f0e5-89a5-4236-b25d-78dd4390d228 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:12.734811 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21f0e5-89a5-4236-b25d-78dd4390d228 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:12.736335 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21f0e5-89a5-4236-b25d-78dd4390d228 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:12.749438 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 632/1259] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:09:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:12.755941 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a18d28-889b-4246-b10e-581f01d7cdbe None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:12.783310 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:12.860530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a18d28-889b-4246-b10e-581f01d7cdbe None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:12.894915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f38ff-ec32-4fec-a2a0-5a4f8ed03563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:12.895524 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f38ff-ec32-4fec-a2a0-5a4f8ed03563 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:12.895805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f38ff-ec32-4fec-a2a0-5a4f8ed03563 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:12.896079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f38ff-ec32-4fec-a2a0-5a4f8ed03563 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:12.896870 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f38ff-ec32-4fec-a2a0-5a4f8ed03563 None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=eced13cc4030423abe63bb1527f6d35a) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:12.897182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f38ff-ec32-4fec-a2a0-5a4f8ed03563 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:12.898747 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f38ff-ec32-4fec-a2a0-5a4f8ed03563 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:12.904641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 628/1260] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:09:12 2022] GET /identity/v3/role_assignments?user.id=eced13cc4030423abe63bb1527f6d35a&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:14.831169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5c593-ed0a-4ee4-ae98-8af02b802a53 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:14.831729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5c593-ed0a-4ee4-ae98-8af02b802a53 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:14.831729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5c593-ed0a-4ee4-ae98-8af02b802a53 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:14.833123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 633/1261] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:14.838649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2b04c-6460-4c55-88b1-a3a796d544c3 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:14.839152 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2b04c-6460-4c55-88b1-a3a796d544c3 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:14.839266 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2b04c-6460-4c55-88b1-a3a796d544c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:14.858706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-bdc2b04c-6460-4c55-88b1-a3a796d544c3 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:14.859942 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-bdc2b04c-6460-4c55-88b1-a3a796d544c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:14.932013 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 629/1262] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:09:15.204127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5c593-ed0a-4ee4-ae98-8af02b802a53 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:15.215844 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:15.308353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5c593-ed0a-4ee4-ae98-8af02b802a53 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:15.344814 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d4bf8-da40-4245-8842-dc2fd5253879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:15.345447 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d4bf8-da40-4245-8842-dc2fd5253879 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:15.345731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d4bf8-da40-4245-8842-dc2fd5253879 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:15.346004 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d4bf8-da40-4245-8842-dc2fd5253879 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:15.356304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d4bf8-da40-4245-8842-dc2fd5253879 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:09:15.361566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-d95d4bf8-da40-4245-8842-dc2fd5253879 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:09:15.361566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:15.361566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:15.361566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:15.361566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:15.361566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:15.361566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:15.361566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:15.361566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:15.361566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:15.361566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:15.361566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:15.361566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:09:15.361566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:09:15.361566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:09:15.361566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:09:15.362767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:09:15.362767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:09:15.362767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:09:15.362767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:09:15.362767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:15.362767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:15.362767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:15.362767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:15.362767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:15.362767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:15.362767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:15.362767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:15.362767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:15.362767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:15.362767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:15.362767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:15.369766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:15.369766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:15.369766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:15.369766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:15.369766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:09:15.369766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:09:15.369766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:09:15.369766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:09:15.369766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:09:15.369766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:09:15.369766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:09:15.369766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:09:15.369766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 634/1263] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:09:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:09:15.370683 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2b04c-6460-4c55-88b1-a3a796d544c3 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:15.388187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:15.468821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2b04c-6460-4c55-88b1-a3a796d544c3 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:15.504444 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50a3e4-fac8-4b06-ade4-2ed78a5300ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:15.505054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50a3e4-fac8-4b06-ade4-2ed78a5300ba None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:15.505328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50a3e4-fac8-4b06-ade4-2ed78a5300ba None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:15.505597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50a3e4-fac8-4b06-ade4-2ed78a5300ba None admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:15.506478 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50a3e4-fac8-4b06-ade4-2ed78a5300ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:15.506780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50a3e4-fac8-4b06-ade4-2ed78a5300ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:15.508168 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50a3e4-fac8-4b06-ade4-2ed78a5300ba None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:15.516364 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 630/1264] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:09:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:15.522245 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d4bf8-da40-4245-8842-dc2fd5253879 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:15.591135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:15.626616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d4bf8-da40-4245-8842-dc2fd5253879 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:15.661352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76b7fe-2e55-476b-9b97-d3b16f94b6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:15.662736 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76b7fe-2e55-476b-9b97-d3b16f94b6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:15.662856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76b7fe-2e55-476b-9b97-d3b16f94b6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:15.663190 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76b7fe-2e55-476b-9b97-d3b16f94b6a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:15.673371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76b7fe-2e55-476b-9b97-d3b16f94b6a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:15.674816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76b7fe-2e55-476b-9b97-d3b16f94b6a7 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:15.690021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-7a76b7fe-2e55-476b-9b97-d3b16f94b6a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:15.690021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:15.690021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:15.690021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:15.690021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:15.690021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:15.690021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:15.690021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:15.690021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:15.690021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:15.690021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:15.690021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:15.690021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:09:15.690021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:09:15.690021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:09:15.690021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:09:15.691232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:15.691232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:15.691232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:15.691232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:15.691232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:15.691232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:15.691232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:15.691232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:15.691232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:15.691232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:15.691232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:15.691232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:15.691232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:15.691232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:15.691232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:15.691232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:15.691232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:09:15.692464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:09:15.692464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:15.692464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:09:15.692464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 635/1265] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:15.697187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50a3e4-fac8-4b06-ade4-2ed78a5300ba None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:15.783573 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:15.803723 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50a3e4-fac8-4b06-ade4-2ed78a5300ba None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:15.837505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4890c-b5e3-4338-95e6-59a5609366d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:15.838112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4890c-b5e3-4338-95e6-59a5609366d9 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:15.838389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4890c-b5e3-4338-95e6-59a5609366d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:15.838657 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4890c-b5e3-4338-95e6-59a5609366d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:15.839318 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4890c-b5e3-4338-95e6-59a5609366d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:15.839859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4890c-b5e3-4338-95e6-59a5609366d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:15.840954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4890c-b5e3-4338-95e6-59a5609366d9 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:15.853689 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 631/1266] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:15.860072 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76b7fe-2e55-476b-9b97-d3b16f94b6a7 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:15.966564 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76b7fe-2e55-476b-9b97-d3b16f94b6a7 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:15.986976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:16.000893 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c07fb8-02eb-4803-b297-bf03bd8833b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:16.001530 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c07fb8-02eb-4803-b297-bf03bd8833b3 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:16.001854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c07fb8-02eb-4803-b297-bf03bd8833b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:16.002173 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c07fb8-02eb-4803-b297-bf03bd8833b3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:16.009857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c07fb8-02eb-4803-b297-bf03bd8833b3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:16.011348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c07fb8-02eb-4803-b297-bf03bd8833b3 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:16.023506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-a9c07fb8-02eb-4803-b297-bf03bd8833b3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:09:16.023506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:16.023506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:16.023506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:16.023506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:16.023506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:16.023506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:16.023506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:16.023506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:16.023506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:16.023506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:16.023506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:16.023506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:09:16.023506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:09:16.023506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:09:16.023506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:16.024653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:16.025795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:16.025795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:16.025795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:09:16.025795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:09:16.025795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:09:16.025795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:09:16.025795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:09:16.025795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:09:16.025795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:09:16.025795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:09:16.025795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 636/1267] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:09:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:09:16.030599 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4890c-b5e3-4338-95e6-59a5609366d9 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:16.133057 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4890c-b5e3-4338-95e6-59a5609366d9 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:16.168257 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c1069-1978-4709-a9a4-15ff889de17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:16.168873 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c1069-1978-4709-a9a4-15ff889de17b None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:16.169178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c1069-1978-4709-a9a4-15ff889de17b None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:16.169446 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c1069-1978-4709-a9a4-15ff889de17b None admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:16.170167 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c1069-1978-4709-a9a4-15ff889de17b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:16.170471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c1069-1978-4709-a9a4-15ff889de17b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:16.171922 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c1069-1978-4709-a9a4-15ff889de17b None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:16.180579 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:16.189078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 632/1268] 10.222.0.59 () {56 vars in 1143 bytes} [Wed Aug 31 19:09:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:16.195666 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c07fb8-02eb-4803-b297-bf03bd8833b3 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:16.299304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c07fb8-02eb-4803-b297-bf03bd8833b3 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:16.335328 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c369303-51d1-4983-b254-c40eff384534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:16.335955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c369303-51d1-4983-b254-c40eff384534 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:16.336229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c369303-51d1-4983-b254-c40eff384534 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:16.336499 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c369303-51d1-4983-b254-c40eff384534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:16.346957 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c369303-51d1-4983-b254-c40eff384534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:16.348417 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c369303-51d1-4983-b254-c40eff384534 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:16.363148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-4c369303-51d1-4983-b254-c40eff384534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:16.363148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:16.363148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:16.363148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:16.363148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:16.363148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:16.363148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:16.363148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:16.363148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:16.363148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:16.363148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:16.363148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:16.363148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:09:16.363148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:09:16.363148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:09:16.363148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:09:16.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:16.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:16.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:16.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:16.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:16.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:16.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:16.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:16.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:16.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:16.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:16.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:16.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:16.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:16.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:16.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:16.364195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:09:16.365117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:09:16.365117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:16.365117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:09:16.365117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 637/1269] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:09:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:16.370567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c1069-1978-4709-a9a4-15ff889de17b None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:16.472027 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c1069-1978-4709-a9a4-15ff889de17b None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:16.507046 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de6f03-a882-4e35-9abc-bd2163d56ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:16.507442 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de6f03-a882-4e35-9abc-bd2163d56ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:16.507702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de6f03-a882-4e35-9abc-bd2163d56ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:16.507972 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de6f03-a882-4e35-9abc-bd2163d56ab8 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:16.508607 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de6f03-a882-4e35-9abc-bd2163d56ab8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:16.508910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de6f03-a882-4e35-9abc-bd2163d56ab8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:16.510263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de6f03-a882-4e35-9abc-bd2163d56ab8 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:16.521650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 633/1270] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:16.527484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c369303-51d1-4983-b254-c40eff384534 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:16.561374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:16.633033 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c369303-51d1-4983-b254-c40eff384534 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:16.668493 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a071c65-b98a-4bf8-aed0-3ee486ef8832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:16.669134 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a071c65-b98a-4bf8-aed0-3ee486ef8832 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:16.669417 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a071c65-b98a-4bf8-aed0-3ee486ef8832 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:16.669693 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a071c65-b98a-4bf8-aed0-3ee486ef8832 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:16.676555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a071c65-b98a-4bf8-aed0-3ee486ef8832 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:16.678602 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a071c65-b98a-4bf8-aed0-3ee486ef8832 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:16.689888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-9a071c65-b98a-4bf8-aed0-3ee486ef8832 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:09:16.689888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:16.689888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:16.689888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:16.689888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:16.689888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:16.689888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:16.689888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:16.689888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:16.689888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:16.689888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:16.689888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:16.689888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:09:16.689888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:09:16.689888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:09:16.692706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:09:16.692706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:16.692706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:16.692706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:16.692706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:16.692706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:16.692706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:16.692706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:16.692706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:16.692706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:16.692706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:16.692706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:16.692706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:16.692706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:16.692706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:16.692706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:16.692706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:16.694719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:09:16.694719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:09:16.694719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:09:16.694719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:09:16.694719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:09:16.694719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:09:16.694719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:09:16.694719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:09:16.694719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 638/1271] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:09:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:16.697876 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de6f03-a882-4e35-9abc-bd2163d56ab8 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:16.758785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:16.801117 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de6f03-a882-4e35-9abc-bd2163d56ab8 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:16.838249 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52410c8a-70ac-474e-8210-ccf69d2d7ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:16.838898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52410c8a-70ac-474e-8210-ccf69d2d7ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:16.839180 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52410c8a-70ac-474e-8210-ccf69d2d7ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:16.839450 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52410c8a-70ac-474e-8210-ccf69d2d7ad7 None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:16.840138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52410c8a-70ac-474e-8210-ccf69d2d7ad7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:16.840440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52410c8a-70ac-474e-8210-ccf69d2d7ad7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:16.841849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52410c8a-70ac-474e-8210-ccf69d2d7ad7 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:16.854600 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 634/1272] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:09:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:16.860865 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a071c65-b98a-4bf8-aed0-3ee486ef8832 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:16.952792 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:16.962330 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a071c65-b98a-4bf8-aed0-3ee486ef8832 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:16.995425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c5991-8ce6-401c-80d0-86c37ceaaa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:16.996116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c5991-8ce6-401c-80d0-86c37ceaaa3b None admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:16.996396 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c5991-8ce6-401c-80d0-86c37ceaaa3b None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:16.996673 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c5991-8ce6-401c-80d0-86c37ceaaa3b None admin] PATH_INFO: `/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/eced13cc4030423abe63bb1527f6d35a/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:16.998596 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c5991-8ce6-401c-80d0-86c37ceaaa3b None admin] RBAC: Authorizing `identity:create_grant(project_id=161c90d34bb34925a1aaeb70c7de7eae, user_id=eced13cc4030423abe63bb1527f6d35a, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:17.000101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c5991-8ce6-401c-80d0-86c37ceaaa3b None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:17.124906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 639/1273] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 31 19:09:16 2022] PUT /identity/v3/projects/161c90d34bb34925a1aaeb70c7de7eae/users/eced13cc4030423abe63bb1527f6d35a/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:09:19.084675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587ef87-9367-4be3-9c19-7adfb3af45ab None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:19.085240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587ef87-9367-4be3-9c19-7adfb3af45ab None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:19.085240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587ef87-9367-4be3-9c19-7adfb3af45ab None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:19.086575 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 635/1274] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:09:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:19.091820 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7112020-48cc-4d8c-bcbd-9ec7a5d2e360 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:19.092268 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7112020-48cc-4d8c-bcbd-9ec7a5d2e360 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:19.092373 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7112020-48cc-4d8c-bcbd-9ec7a5d2e360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:19.111784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-b7112020-48cc-4d8c-bcbd-9ec7a5d2e360 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:19.113031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-b7112020-48cc-4d8c-bcbd-9ec7a5d2e360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:19.181289 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 640/1275] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:09:19.455089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587ef87-9367-4be3-9c19-7adfb3af45ab None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:19.557770 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587ef87-9367-4be3-9c19-7adfb3af45ab None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:19.574936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:19.594059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca240d4-3cf0-4a5d-98a8-a2e085853de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:19.594697 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca240d4-3cf0-4a5d-98a8-a2e085853de6 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:19.595004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca240d4-3cf0-4a5d-98a8-a2e085853de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:19.595279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca240d4-3cf0-4a5d-98a8-a2e085853de6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:19.606572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca240d4-3cf0-4a5d-98a8-a2e085853de6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:09:19.611837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-0ca240d4-3cf0-4a5d-98a8-a2e085853de6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:09:19.611837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:19.611837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:19.611837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:19.611837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:19.611837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:19.611837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:19.611837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:19.611837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:19.611837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:19.611837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:19.611837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:19.611837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:09:19.611837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:09:19.611837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:09:19.611837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:09:19.612999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:09:19.612999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:09:19.612999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:09:19.612999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:09:19.612999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:19.612999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:19.612999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:19.612999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:19.612999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:19.612999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:19.612999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:19.612999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:19.612999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:19.612999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:19.612999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:19.612999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:19.614097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:19.614097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:19.614097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:19.614097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:19.614097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:09:19.614097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:09:19.614097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:09:19.614097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:09:19.614097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:09:19.614097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:09:19.614097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:09:19.614097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:19.618072 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 636/1276] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 31 19:09:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:19.619228 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7112020-48cc-4d8c-bcbd-9ec7a5d2e360 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:19.700774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:19.718840 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7112020-48cc-4d8c-bcbd-9ec7a5d2e360 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:19.754961 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f98a6-125f-4395-9599-c87c1bd8f471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:19.755689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f98a6-125f-4395-9599-c87c1bd8f471 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:19.755970 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f98a6-125f-4395-9599-c87c1bd8f471 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:19.756247 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f98a6-125f-4395-9599-c87c1bd8f471 None admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:19.757165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f98a6-125f-4395-9599-c87c1bd8f471 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:19.757362 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f98a6-125f-4395-9599-c87c1bd8f471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:19.758773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f98a6-125f-4395-9599-c87c1bd8f471 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:19.766768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 641/1277] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 31 19:09:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:19.773319 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca240d4-3cf0-4a5d-98a8-a2e085853de6 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:19.872715 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca240d4-3cf0-4a5d-98a8-a2e085853de6 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:19.906553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2159b9-1350-4623-854d-0533e1cf71a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:19.907201 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2159b9-1350-4623-854d-0533e1cf71a1 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:19.907478 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2159b9-1350-4623-854d-0533e1cf71a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:19.907747 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2159b9-1350-4623-854d-0533e1cf71a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:19.917734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2159b9-1350-4623-854d-0533e1cf71a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:19.919318 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2159b9-1350-4623-854d-0533e1cf71a1 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:19.934182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-bc2159b9-1350-4623-854d-0533e1cf71a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:19.934182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:19.934182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:19.934182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:19.934182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:19.934182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:19.934182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:19.934182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:19.934182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:19.934182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:19.934182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:19.934182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:19.934182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:09:19.934182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:09:19.934182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:09:19.934182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:09:19.935378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:19.935378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:19.935378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:19.935378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:19.935378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:19.935378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:19.935378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:19.935378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:19.935378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:19.935378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:19.935378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:19.935378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:19.935378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:19.935378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:19.935378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:19.935378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:19.935378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:09:19.936358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:09:19.936358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:19.936358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:19.936358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 637/1278] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:09:19.941501 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f98a6-125f-4395-9599-c87c1bd8f471 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:20.043853 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f98a6-125f-4395-9599-c87c1bd8f471 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:20.077090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204caa00-e715-4196-addb-9b658f55eb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:20.077737 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204caa00-e715-4196-addb-9b658f55eb94 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:20.078011 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204caa00-e715-4196-addb-9b658f55eb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:20.078310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204caa00-e715-4196-addb-9b658f55eb94 None admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:20.079012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204caa00-e715-4196-addb-9b658f55eb94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:20.079311 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204caa00-e715-4196-addb-9b658f55eb94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:20.080668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204caa00-e715-4196-addb-9b658f55eb94 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:20.092928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 642/1279] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:20.098771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2159b9-1350-4623-854d-0533e1cf71a1 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:20.131202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:20.203739 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2159b9-1350-4623-854d-0533e1cf71a1 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:20.237761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944fb43c-ef41-4c6d-a4c8-887c2c8d6dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:20.238388 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fb43c-ef41-4c6d-a4c8-887c2c8d6dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:20.238665 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fb43c-ef41-4c6d-a4c8-887c2c8d6dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:20.239021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fb43c-ef41-4c6d-a4c8-887c2c8d6dc1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:20.247190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944fb43c-ef41-4c6d-a4c8-887c2c8d6dc1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:20.248618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944fb43c-ef41-4c6d-a4c8-887c2c8d6dc1 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:20.260647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-944fb43c-ef41-4c6d-a4c8-887c2c8d6dc1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:09:20.260647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:20.260647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:20.260647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:20.260647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:20.260647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:20.260647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:20.260647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:20.260647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:20.260647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:20.260647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:20.260647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:20.260647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:09:20.260647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:09:20.260647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:09:20.260647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:20.261843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:20.263187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:20.263187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:20.263187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:09:20.263187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:09:20.263187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:09:20.263187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:09:20.263187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:09:20.263187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:09:20.263187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:09:20.263187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:20.263187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 638/1280] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:09:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:20.267589 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204caa00-e715-4196-addb-9b658f55eb94 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:20.271956 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:20.365718 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204caa00-e715-4196-addb-9b658f55eb94 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:20.399469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa94e2-c6ae-416e-b25b-5c47b66f1149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:20.400073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa94e2-c6ae-416e-b25b-5c47b66f1149 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:20.400356 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa94e2-c6ae-416e-b25b-5c47b66f1149 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:20.400629 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa94e2-c6ae-416e-b25b-5c47b66f1149 None admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:20.401355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa94e2-c6ae-416e-b25b-5c47b66f1149 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:20.401662 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa94e2-c6ae-416e-b25b-5c47b66f1149 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:20.403143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa94e2-c6ae-416e-b25b-5c47b66f1149 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:20.419939 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 643/1281] 10.222.0.59 () {56 vars in 1143 bytes} [Wed Aug 31 19:09:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:20.425635 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944fb43c-ef41-4c6d-a4c8-887c2c8d6dc1 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:20.518571 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:20.528716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944fb43c-ef41-4c6d-a4c8-887c2c8d6dc1 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:20.562452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abc6fe-11b9-47d5-903d-66f47d30a740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:20.563086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abc6fe-11b9-47d5-903d-66f47d30a740 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:20.563364 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abc6fe-11b9-47d5-903d-66f47d30a740 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:20.563660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abc6fe-11b9-47d5-903d-66f47d30a740 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:20.631546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abc6fe-11b9-47d5-903d-66f47d30a740 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:20.632957 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abc6fe-11b9-47d5-903d-66f47d30a740 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:20.721563 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:20.732568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 639/1282] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 31 19:09:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:09:20.738399 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa94e2-c6ae-416e-b25b-5c47b66f1149 None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:20.799530 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:20.838189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa94e2-c6ae-416e-b25b-5c47b66f1149 None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:20.871739 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bd3a4-e772-4054-95bf-40da6e1ff14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:20.872374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bd3a4-e772-4054-95bf-40da6e1ff14c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:20.872654 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bd3a4-e772-4054-95bf-40da6e1ff14c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:20.872923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bd3a4-e772-4054-95bf-40da6e1ff14c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:20.883769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bd3a4-e772-4054-95bf-40da6e1ff14c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:20.885159 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bd3a4-e772-4054-95bf-40da6e1ff14c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:20.900457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-762bd3a4-e772-4054-95bf-40da6e1ff14c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:20.900457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:20.900457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:20.900457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:20.900457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:20.900457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:20.900457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:20.900457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:20.900457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:20.900457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:20.900457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:20.900457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:20.900457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:09:20.900457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:09:20.900457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:09:20.900457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:09:20.901534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:20.901534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:20.901534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:20.901534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:20.901534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:20.901534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:20.901534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:20.901534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:20.901534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:20.901534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:20.901534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:20.901534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:20.901534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:20.901534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:20.901534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:20.901534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:20.901534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:09:20.902519 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:09:20.902519 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:09:20.902519 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:09:20.902519 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 644/1283] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:09:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:20.907417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abc6fe-11b9-47d5-903d-66f47d30a740 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:21.008601 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abc6fe-11b9-47d5-903d-66f47d30a740 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:21.042771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f3890-8eaa-43dc-b94c-289ce03c5ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:21.043450 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f3890-8eaa-43dc-b94c-289ce03c5ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:21.043731 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f3890-8eaa-43dc-b94c-289ce03c5ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:21.044006 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f3890-8eaa-43dc-b94c-289ce03c5ed2 None admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:21.044647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f3890-8eaa-43dc-b94c-289ce03c5ed2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:21.044950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f3890-8eaa-43dc-b94c-289ce03c5ed2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:21.046306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f3890-8eaa-43dc-b94c-289ce03c5ed2 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:21.058428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 640/1284] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 31 19:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:21.064170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bd3a4-e772-4054-95bf-40da6e1ff14c None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:21.165598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bd3a4-e772-4054-95bf-40da6e1ff14c None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:21.179752 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:21.199847 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6a63f-ab96-4b17-9b3f-1edde9544e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:21.200478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6a63f-ab96-4b17-9b3f-1edde9544e7f None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:21.200756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6a63f-ab96-4b17-9b3f-1edde9544e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:21.201030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6a63f-ab96-4b17-9b3f-1edde9544e7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:21.207443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6a63f-ab96-4b17-9b3f-1edde9544e7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:21.209183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6a63f-ab96-4b17-9b3f-1edde9544e7f None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:21.220264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application [None req-b2a6a63f-ab96-4b17-9b3f-1edde9544e7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:09:21.220264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:21.220264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:21.220264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:21.220264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:21.220264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:21.220264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:21.220264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:21.220264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:21.220264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:21.220264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:21.220264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:21.220264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:09:21.220264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:09:21.220264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:09:21.221349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:09:21.221349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:21.221349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:21.221349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:21.221349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:21.221349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:21.221349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:21.221349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:21.221349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:21.221349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:21.221349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:21.221349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:21.221349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:21.221349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:21.221349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:21.221349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:21.221349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:21.222702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:09:21.222702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:09:21.222702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:09:21.222702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:09:21.222702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:09:21.222702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:09:21.222702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:09:21.222702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: ERROR keystone.server.flask.application Aug 31 19:09:21.222702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 645/1285] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 31 19:09:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:21.227284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f3890-8eaa-43dc-b94c-289ce03c5ed2 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:21.288133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:21.343468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f3890-8eaa-43dc-b94c-289ce03c5ed2 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:21.381941 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32600d8c-d179-416d-b539-59b1c34e018a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:21.382566 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32600d8c-d179-416d-b539-59b1c34e018a None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:21.382846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32600d8c-d179-416d-b539-59b1c34e018a None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:21.383150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32600d8c-d179-416d-b539-59b1c34e018a None admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:21.383912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32600d8c-d179-416d-b539-59b1c34e018a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:21.384223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32600d8c-d179-416d-b539-59b1c34e018a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:21.385681 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32600d8c-d179-416d-b539-59b1c34e018a None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:21.399826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 641/1286] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 31 19:09:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:21.406021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6a63f-ab96-4b17-9b3f-1edde9544e7f None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:21.523381 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6a63f-ab96-4b17-9b3f-1edde9544e7f None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:21.560985 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd59813-a8d4-4f0f-84eb-9f5e15209d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:21.561593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd59813-a8d4-4f0f-84eb-9f5e15209d35 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:21.561891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd59813-a8d4-4f0f-84eb-9f5e15209d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:21.562131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd59813-a8d4-4f0f-84eb-9f5e15209d35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:21.562852 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd59813-a8d4-4f0f-84eb-9f5e15209d35 None admin] RBAC: Adding query filter params (role.id=929a670f39fc4a97af9d8ab3f9b5e91a, scope.project.id=161c90d34bb34925a1aaeb70c7de7eae, user.id=eced13cc4030423abe63bb1527f6d35a) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:21.563189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd59813-a8d4-4f0f-84eb-9f5e15209d35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:21.564613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd59813-a8d4-4f0f-84eb-9f5e15209d35 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:21.570621 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 646/1287] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 31 19:09:21 2022] GET /identity/v3/role_assignments?user.id=eced13cc4030423abe63bb1527f6d35a&scope.project.id=161c90d34bb34925a1aaeb70c7de7eae&role.id=929a670f39fc4a97af9d8ab3f9b5e91a => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:23.511727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bae1ed-babf-41fc-a72c-3d66f14cfbd9 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:23.512308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bae1ed-babf-41fc-a72c-3d66f14cfbd9 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:23.512308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bae1ed-babf-41fc-a72c-3d66f14cfbd9 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:23.513620 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 642/1288] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:23.518612 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a09ad-6d72-4dc8-9aa5-2b88ef59c4b4 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:23.519059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a09ad-6d72-4dc8-9aa5-2b88ef59c4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:23.520052 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a09ad-6d72-4dc8-9aa5-2b88ef59c4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:23.541540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-7c9a09ad-6d72-4dc8-9aa5-2b88ef59c4b4 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:23.541540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7c9a09ad-6d72-4dc8-9aa5-2b88ef59c4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:23.575312 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:23.611705 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 647/1289] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:09:23.881377 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bae1ed-babf-41fc-a72c-3d66f14cfbd9 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:23.986765 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bae1ed-babf-41fc-a72c-3d66f14cfbd9 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:23.997781 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:24.020716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9838e7e-e01f-4591-ad16-db10f4021a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:24.021830 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9838e7e-e01f-4591-ad16-db10f4021a60 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:24.022186 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9838e7e-e01f-4591-ad16-db10f4021a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:24.022459 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9838e7e-e01f-4591-ad16-db10f4021a60 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:24.023062 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9838e7e-e01f-4591-ad16-db10f4021a60 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:24.024414 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9838e7e-e01f-4591-ad16-db10f4021a60 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:24.032917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-f9838e7e-e01f-4591-ad16-db10f4021a60 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:09:24.032917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:24.032917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:09:24.032917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:09:24.032917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:09:24.032917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:09:24.032917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:09:24.032917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:09:24.032917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:09:24.032917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:24.032917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:09:24.032917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:24.032917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:24.032917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:24.032917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:24.032917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:24.032917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:24.034135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:24.034135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:24.034135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:24.034135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:24.034135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:24.034135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:24.034135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:09:24.034135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:09:24.034135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:09:24.034135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:09:24.034135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:24.034135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:24.034135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:24.034135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:24.034135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:24.034135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:24.035199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:24.035199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:24.035199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:24.035199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:24.035199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:24.035199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:24.035199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:24.035199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:24.035199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:24.035199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:24.035199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:09:24.035199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:09:24.035199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:09:24.035199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:24.035199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 643/1290] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 31 19:09:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:24.040973 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a09ad-6d72-4dc8-9aa5-2b88ef59c4b4 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:24.143917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a09ad-6d72-4dc8-9aa5-2b88ef59c4b4 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:24.178407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77256f6a-9069-4cd5-977c-ae5f63378f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:24.179047 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77256f6a-9069-4cd5-977c-ae5f63378f2a None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:24.179333 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77256f6a-9069-4cd5-977c-ae5f63378f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:24.179606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77256f6a-9069-4cd5-977c-ae5f63378f2a None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:24.180236 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77256f6a-9069-4cd5-977c-ae5f63378f2a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:24.180541 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77256f6a-9069-4cd5-977c-ae5f63378f2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:24.181945 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77256f6a-9069-4cd5-977c-ae5f63378f2a None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:24.186722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 648/1291] 10.222.0.59 () {56 vars in 1116 bytes} [Wed Aug 31 19:09:24 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:24.192582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9838e7e-e01f-4591-ad16-db10f4021a60 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:24.298294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9838e7e-e01f-4591-ad16-db10f4021a60 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:24.333259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc46d6-a1cd-4b4f-a917-1e1b9b5b7e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:24.334067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc46d6-a1cd-4b4f-a917-1e1b9b5b7e43 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:24.334347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc46d6-a1cd-4b4f-a917-1e1b9b5b7e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:24.334716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc46d6-a1cd-4b4f-a917-1e1b9b5b7e43 None admin] PATH_INFO: `/v3/services` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:24.335325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc46d6-a1cd-4b4f-a917-1e1b9b5b7e43 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:24.335740 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc46d6-a1cd-4b4f-a917-1e1b9b5b7e43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:24.337016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc46d6-a1cd-4b4f-a917-1e1b9b5b7e43 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:24.343168 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 644/1292] 10.222.0.59 () {56 vars in 1116 bytes} [Wed Aug 31 19:09:24 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:26.291066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2ca2b-4e62-4053-966c-6248a70183ed None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:26.291588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2ca2b-4e62-4053-966c-6248a70183ed None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:26.291588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2ca2b-4e62-4053-966c-6248a70183ed None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:26.292920 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 649/1293] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:26.298292 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f673d4f-4e77-430e-b566-4866b0ac63ca None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:26.299769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f673d4f-4e77-430e-b566-4866b0ac63ca None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:26.299769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f673d4f-4e77-430e-b566-4866b0ac63ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:26.317168 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-8f673d4f-4e77-430e-b566-4866b0ac63ca None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:26.318459 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8f673d4f-4e77-430e-b566-4866b0ac63ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:26.348586 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:26.386147 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 645/1294] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:09:26.650648 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2ca2b-4e62-4053-966c-6248a70183ed None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:26.681109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:26.750640 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2ca2b-4e62-4053-966c-6248a70183ed None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:26.785474 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217c4c5-b263-4a2b-927a-240c1a97fbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:26.786096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217c4c5-b263-4a2b-927a-240c1a97fbfe None admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:26.787159 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217c4c5-b263-4a2b-927a-240c1a97fbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:26.787923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217c4c5-b263-4a2b-927a-240c1a97fbfe None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:26.788076 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217c4c5-b263-4a2b-927a-240c1a97fbfe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:26.789490 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217c4c5-b263-4a2b-927a-240c1a97fbfe None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:26.867956 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 650/1295] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 31 19:09:26 2022] POST /identity/v3/services => generated 246 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:09:28.824760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43fad79-ff1e-48f4-821c-1bdb4bf7846c None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:28.825324 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43fad79-ff1e-48f4-821c-1bdb4bf7846c None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:28.825324 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43fad79-ff1e-48f4-821c-1bdb4bf7846c None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:28.826684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 646/1296] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:28.832103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ed80a-c25a-4933-b520-24d0b86a805d None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:28.838180 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ed80a-c25a-4933-b520-24d0b86a805d None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:28.838180 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ed80a-c25a-4933-b520-24d0b86a805d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:28.851290 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-f40ed80a-c25a-4933-b520-24d0b86a805d None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:28.852563 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f40ed80a-c25a-4933-b520-24d0b86a805d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:28.925853 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 651/1297] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:28 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:09:29.202839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43fad79-ff1e-48f4-821c-1bdb4bf7846c None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:29.306386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43fad79-ff1e-48f4-821c-1bdb4bf7846c None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:29.340621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0936e-eac7-4be0-84c8-279c75e9550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:29.341269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0936e-eac7-4be0-84c8-279c75e9550e None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:29.341557 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0936e-eac7-4be0-84c8-279c75e9550e None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:29.341828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0936e-eac7-4be0-84c8-279c75e9550e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:29.342406 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c0936e-eac7-4be0-84c8-279c75e9550e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:29.343794 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c0936e-eac7-4be0-84c8-279c75e9550e None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:29.352751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-e4c0936e-eac7-4be0-84c8-279c75e9550e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:09:29.352751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:29.352751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:09:29.352751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:09:29.352751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:09:29.352751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:09:29.352751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:09:29.352751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:09:29.352751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:09:29.352751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:29.352751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:09:29.352751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:29.352751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:29.352751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:29.352751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:29.352751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:29.352751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:29.353940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:29.353940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:29.353940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:29.353940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:29.353940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:29.353940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:29.353940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:09:29.353940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:09:29.353940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:09:29.353940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:09:29.353940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:29.353940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:29.353940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:29.353940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:29.353940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:29.353940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:29.354842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:29.354842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:29.354842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:29.354842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:29.354842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:29.354842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:29.354842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:29.354842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:29.354842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:29.354842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:29.354842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:09:29.354842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:09:29.354842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:09:29.354842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:29.354842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 647/1298] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 31 19:09:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:29.362333 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ed80a-c25a-4933-b520-24d0b86a805d None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:29.382860 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:29.467376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ed80a-c25a-4933-b520-24d0b86a805d None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:29.503672 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec765e-ede3-435b-acca-322eb84b11fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:29.504287 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec765e-ede3-435b-acca-322eb84b11fb None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:29.504604 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec765e-ede3-435b-acca-322eb84b11fb None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:29.504886 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec765e-ede3-435b-acca-322eb84b11fb None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:29.506183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec765e-ede3-435b-acca-322eb84b11fb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:29.506601 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec765e-ede3-435b-acca-322eb84b11fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:29.508023 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec765e-ede3-435b-acca-322eb84b11fb None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:29.513256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 652/1299] 10.222.0.59 () {56 vars in 1116 bytes} [Wed Aug 31 19:09:29 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:29.519336 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0936e-eac7-4be0-84c8-279c75e9550e None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:29.551721 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:29.619887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0936e-eac7-4be0-84c8-279c75e9550e None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:29.656523 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b774f7-1d27-49e9-b650-9e698f7d4eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:29.657141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b774f7-1d27-49e9-b650-9e698f7d4eee None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:29.657470 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b774f7-1d27-49e9-b650-9e698f7d4eee None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:29.657717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b774f7-1d27-49e9-b650-9e698f7d4eee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:29.658420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b774f7-1d27-49e9-b650-9e698f7d4eee None admin] RBAC: Adding query filter params (interface=public, service_id=2a0bd823080e4336ab465a354a00d74b, region_id=RegionOne) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:29.658731 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b774f7-1d27-49e9-b650-9e698f7d4eee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:29.660138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b774f7-1d27-49e9-b650-9e698f7d4eee None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:29.669117 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 648/1300] 10.222.0.59 () {56 vars in 1251 bytes} [Wed Aug 31 19:09:29 2022] GET /identity/v3/endpoints?service_id=2a0bd823080e4336ab465a354a00d74b&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:09:29.676053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec765e-ede3-435b-acca-322eb84b11fb None admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:29.760147 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:29.778588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec765e-ede3-435b-acca-322eb84b11fb None admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:29.812747 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274a17b-2cb8-4c3d-bedd-24d2aef7665c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:29.820225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274a17b-2cb8-4c3d-bedd-24d2aef7665c None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:29.820225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274a17b-2cb8-4c3d-bedd-24d2aef7665c None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:29.820225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274a17b-2cb8-4c3d-bedd-24d2aef7665c None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:29.820225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274a17b-2cb8-4c3d-bedd-24d2aef7665c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:29.820225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274a17b-2cb8-4c3d-bedd-24d2aef7665c None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:29.823530 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 653/1301] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 31 19:09:29 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:09:31.734094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbf947-0a36-458f-934e-be2b000609f1 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:31.734638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbf947-0a36-458f-934e-be2b000609f1 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:31.734638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbf947-0a36-458f-934e-be2b000609f1 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:31.736002 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 649/1302] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:31.741724 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1eb656-3c91-4024-8bcb-24bd11da47e1 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:31.742159 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1eb656-3c91-4024-8bcb-24bd11da47e1 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:31.742374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1eb656-3c91-4024-8bcb-24bd11da47e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:31.761501 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-1b1eb656-3c91-4024-8bcb-24bd11da47e1 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:31.762705 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-1b1eb656-3c91-4024-8bcb-24bd11da47e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:31.835363 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 654/1303] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:31 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:09:32.131930 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbf947-0a36-458f-934e-be2b000609f1 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:32.248718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:32.256057 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbf947-0a36-458f-934e-be2b000609f1 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:32.293292 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1aa162-0f72-4df5-a17f-1f8e6fac5a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:32.293922 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aa162-0f72-4df5-a17f-1f8e6fac5a05 None admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:32.294201 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aa162-0f72-4df5-a17f-1f8e6fac5a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:32.294472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aa162-0f72-4df5-a17f-1f8e6fac5a05 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:32.295092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1aa162-0f72-4df5-a17f-1f8e6fac5a05 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:32.296455 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1aa162-0f72-4df5-a17f-1f8e6fac5a05 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:32.305432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application [None req-9e1aa162-0f72-4df5-a17f-1f8e6fac5a05 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:09:32.305432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:32.305432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:09:32.305432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:09:32.305432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:09:32.305432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:09:32.305432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:09:32.305432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:09:32.305432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:09:32.305432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:32.305432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:09:32.305432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:32.305432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:09:32.305432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:09:32.305432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:09:32.305432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:09:32.305432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:09:32.309287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:09:32.309287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:09:32.309287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:09:32.309287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:09:32.309287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:09:32.309287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:09:32.309287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:09:32.309287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:09:32.309287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:09:32.309287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:09:32.309287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:09:32.309287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:09:32.309287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:09:32.309287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:09:32.309287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:09:32.309287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:09:32.310411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:09:32.310411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application with Lock( Aug 31 19:09:32.310411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:09:32.310411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:09:32.310411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:09:32.310411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:09:32.310411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:09:32.310411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:09:32.310411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:09:32.310411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:09:32.310411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:09:32.310411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:09:32.310411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:09:32.310411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: ERROR keystone.server.flask.application Aug 31 19:09:32.310411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 650/1304] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 31 19:09:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:09:32.312650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1eb656-3c91-4024-8bcb-24bd11da47e1 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:32.397166 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:32.433034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1eb656-3c91-4024-8bcb-24bd11da47e1 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:32.466802 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf419ddf-55cc-4952-8f2e-73e913386d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:32.467435 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf419ddf-55cc-4952-8f2e-73e913386d37 None admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:32.467715 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf419ddf-55cc-4952-8f2e-73e913386d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:32.467986 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf419ddf-55cc-4952-8f2e-73e913386d37 None admin] PATH_INFO: `/v3/services` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:32.468614 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf419ddf-55cc-4952-8f2e-73e913386d37 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:09:32.468915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf419ddf-55cc-4952-8f2e-73e913386d37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:32.470319 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf419ddf-55cc-4952-8f2e-73e913386d37 None admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:32.475321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 655/1305] 10.222.0.59 () {56 vars in 1116 bytes} [Wed Aug 31 19:09:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:09:32.481625 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1aa162-0f72-4df5-a17f-1f8e6fac5a05 None admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:32.588433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1aa162-0f72-4df5-a17f-1f8e6fac5a05 None admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:32.622928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8a38c-05fd-4ac5-96c4-9864649bd178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:32.623542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8a38c-05fd-4ac5-96c4-9864649bd178 None admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:32.623825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8a38c-05fd-4ac5-96c4-9864649bd178 None admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:32.624100 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8a38c-05fd-4ac5-96c4-9864649bd178 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:32.624763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e8a38c-05fd-4ac5-96c4-9864649bd178 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:32.626144 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e8a38c-05fd-4ac5-96c4-9864649bd178 None admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:32.702614 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 651/1306] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:09:32 2022] POST /identity/v3/endpoints => generated 331 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:09:42.501996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=30 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:42.697853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:33.389311 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12841d88-9823-4d1e-8010-330f2bf63b9d None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:33.389904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12841d88-9823-4d1e-8010-330f2bf63b9d None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:33.389904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12841d88-9823-4d1e-8010-330f2bf63b9d None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:33.397061 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9e72e-e784-4af4-925a-f9fb2e60f771 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:33.397618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9e72e-e784-4af4-925a-f9fb2e60f771 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:33.397618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9e72e-e784-4af4-925a-f9fb2e60f771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:33.402661 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 656/1307] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:16:33 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:16:33.502745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-d4f9e72e-e784-4af4-925a-f9fb2e60f771 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:33.502745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d4f9e72e-e784-4af4-925a-f9fb2e60f771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:33.640011 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 652/1308] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:33.932359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a320163-6868-4f0f-a40c-4a9c7768396f None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:33.932687 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a320163-6868-4f0f-a40c-4a9c7768396f None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:33.932974 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a320163-6868-4f0f-a40c-4a9c7768396f None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:33.934526 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 657/1309] 10.222.0.59 () {54 vars in 965 bytes} [Wed Aug 31 19:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:16:33.941056 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48af873-e616-496f-8171-57a0a5c6dd24 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:33.941714 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48af873-e616-496f-8171-57a0a5c6dd24 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:33.942815 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48af873-e616-496f-8171-57a0a5c6dd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:34.027625 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-a48af873-e616-496f-8171-57a0a5c6dd24 None None] MFA Rules not processed for user `b7a51545146846cc8284b7044b32198a`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:34.029140 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a48af873-e616-496f-8171-57a0a5c6dd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:34.064819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:34.144682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 653/1310] 10.222.0.59 () {58 vars in 1062 bytes} [Wed Aug 31 19:16:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:34.154258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a320163-6868-4f0f-a40c-4a9c7768396f None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:34.156580 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8a320163-6868-4f0f-a40c-4a9c7768396f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:34.299778 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a320163-6868-4f0f-a40c-4a9c7768396f None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:34.381765 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239dea37-4860-49bc-b63d-443bd45aa8a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:34.382388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dea37-4860-49bc-b63d-443bd45aa8a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:34.382673 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dea37-4860-49bc-b63d-443bd45aa8a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:34.382985 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dea37-4860-49bc-b63d-443bd45aa8a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:34.384610 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-239dea37-4860-49bc-b63d-443bd45aa8a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:34.444571 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:34.504019 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239dea37-4860-49bc-b63d-443bd45aa8a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:34.506367 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239dea37-4860-49bc-b63d-443bd45aa8a8 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:34.530044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 658/1311] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:16:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:16:36.799947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe52cc0-955a-4a78-8733-cd53565331a8 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:36.800529 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe52cc0-955a-4a78-8733-cd53565331a8 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:36.800529 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe52cc0-955a-4a78-8733-cd53565331a8 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:36.801918 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 654/1312] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:16:36.807166 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fee32-76b2-41e5-bcaa-4182b1603f72 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:36.807617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fee32-76b2-41e5-bcaa-4182b1603f72 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:36.807711 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fee32-76b2-41e5-bcaa-4182b1603f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:36.883985 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-952fee32-76b2-41e5-bcaa-4182b1603f72 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:36.885231 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-952fee32-76b2-41e5-bcaa-4182b1603f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:36.993063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 659/1313] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:37.288567 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe52cc0-955a-4a78-8733-cd53565331a8 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:37.290104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-0fe52cc0-955a-4a78-8733-cd53565331a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:37.403258 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe52cc0-955a-4a78-8733-cd53565331a8 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:37.475988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:37.478684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b325ac7-56d3-458d-8017-c2ea82461dd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:37.479334 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b325ac7-56d3-458d-8017-c2ea82461dd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:37.479614 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b325ac7-56d3-458d-8017-c2ea82461dd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:37.479920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b325ac7-56d3-458d-8017-c2ea82461dd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:37.481623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4b325ac7-56d3-458d-8017-c2ea82461dd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:37.596962 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b325ac7-56d3-458d-8017-c2ea82461dd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:37.598434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b325ac7-56d3-458d-8017-c2ea82461dd1 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:37.617895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 655/1314] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:16:40.201854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9f818-f941-4702-b6e8-1402b64abf79 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:40.202442 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9f818-f941-4702-b6e8-1402b64abf79 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:40.202540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9f818-f941-4702-b6e8-1402b64abf79 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:40.203888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 660/1315] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:16:40.209755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66ec2b-8560-4cf1-af3c-89978a7f5236 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:40.210181 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66ec2b-8560-4cf1-af3c-89978a7f5236 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:40.210295 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66ec2b-8560-4cf1-af3c-89978a7f5236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:40.278401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-8f66ec2b-8560-4cf1-af3c-89978a7f5236 None None] MFA Rules not processed for user `7404abca54674994bf5e6ec8f024692d`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:40.279676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8f66ec2b-8560-4cf1-af3c-89978a7f5236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:40.331996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:40.376658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 656/1316] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:16:40 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:40.658047 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9f818-f941-4702-b6e8-1402b64abf79 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:40.659618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d6a9f818-f941-4702-b6e8-1402b64abf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:40.669860 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:40.764793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9f818-f941-4702-b6e8-1402b64abf79 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:40.836227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc0e73-cb21-49df-837b-a2213edb01b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:40.836829 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc0e73-cb21-49df-837b-a2213edb01b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:40.837107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc0e73-cb21-49df-837b-a2213edb01b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:40.837376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc0e73-cb21-49df-837b-a2213edb01b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:40.838976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ffdc0e73-cb21-49df-837b-a2213edb01b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:40.900729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:40.946013 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc0e73-cb21-49df-837b-a2213edb01b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:40.947403 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc0e73-cb21-49df-837b-a2213edb01b5 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:40.966172 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 661/1317] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:16:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:16:43.863206 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5efaa-eee9-4cd2-8718-73e24401ae07 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:43.863784 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5efaa-eee9-4cd2-8718-73e24401ae07 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:43.863784 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5efaa-eee9-4cd2-8718-73e24401ae07 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:43.865506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 657/1318] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:16:43.870683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3db6d-b600-4a52-bdf6-b5ca3f2ba130 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:43.871170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3db6d-b600-4a52-bdf6-b5ca3f2ba130 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:43.871266 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3db6d-b600-4a52-bdf6-b5ca3f2ba130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:43.949456 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-f4b3db6d-b600-4a52-bdf6-b5ca3f2ba130 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:43.953305 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f4b3db6d-b600-4a52-bdf6-b5ca3f2ba130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:44.050009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 662/1319] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:16:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:44.364492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164653d4-4e8c-40ae-934e-4ca41fe08d0b None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:44.364795 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164653d4-4e8c-40ae-934e-4ca41fe08d0b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:44.365072 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164653d4-4e8c-40ae-934e-4ca41fe08d0b None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:44.366459 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 658/1320] 10.222.0.59 () {54 vars in 965 bytes} [Wed Aug 31 19:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:16:44.372463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb190e12-7661-44ac-ac94-0fecc7186731 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:44.372756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb190e12-7661-44ac-ac94-0fecc7186731 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:44.373028 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb190e12-7661-44ac-ac94-0fecc7186731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:44.395868 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:44.451405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-cb190e12-7661-44ac-ac94-0fecc7186731 None None] MFA Rules not processed for user `b7a51545146846cc8284b7044b32198a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:44.452619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-cb190e12-7661-44ac-ac94-0fecc7186731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:44.530135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 663/1321] 10.222.0.59 () {58 vars in 1062 bytes} [Wed Aug 31 19:16:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:16:44.539419 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164653d4-4e8c-40ae-934e-4ca41fe08d0b None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:44.540938 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-164653d4-4e8c-40ae-934e-4ca41fe08d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:44.644188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164653d4-4e8c-40ae-934e-4ca41fe08d0b None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:44.719192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c935be-15ec-483c-b17e-d4b64bf069b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:44.719804 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c935be-15ec-483c-b17e-d4b64bf069b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:44.720085 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c935be-15ec-483c-b17e-d4b64bf069b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:44.720361 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c935be-15ec-483c-b17e-d4b64bf069b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:44.721945 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-79c935be-15ec-483c-b17e-d4b64bf069b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:44.728893 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:44.836537 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c935be-15ec-483c-b17e-d4b64bf069b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:44.837973 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c935be-15ec-483c-b17e-d4b64bf069b5 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:44.856116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 659/1322] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:16:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:16:48.772289 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf229ca5-2d5e-4c9c-911d-257a52f01841 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:48.772837 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf229ca5-2d5e-4c9c-911d-257a52f01841 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:48.772837 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf229ca5-2d5e-4c9c-911d-257a52f01841 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:48.774193 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 664/1323] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:16:48.779577 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946532f2-a179-4fff-be8c-affd3a45c26c None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:48.780042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946532f2-a179-4fff-be8c-affd3a45c26c None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:48.780148 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946532f2-a179-4fff-be8c-affd3a45c26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:48.851949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-946532f2-a179-4fff-be8c-affd3a45c26c None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:48.853214 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-946532f2-a179-4fff-be8c-affd3a45c26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:48.914564 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:48.957295 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 660/1324] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:49.252627 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf229ca5-2d5e-4c9c-911d-257a52f01841 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:49.254806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-bf229ca5-2d5e-4c9c-911d-257a52f01841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:49.374925 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf229ca5-2d5e-4c9c-911d-257a52f01841 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:49.388351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:49.454690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e43c03-2535-4cbd-93b7-f85c5086d245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:49.455348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e43c03-2535-4cbd-93b7-f85c5086d245 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:49.455626 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e43c03-2535-4cbd-93b7-f85c5086d245 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:49.455892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e43c03-2535-4cbd-93b7-f85c5086d245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:49.457495 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-19e43c03-2535-4cbd-93b7-f85c5086d245 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:49.569220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e43c03-2535-4cbd-93b7-f85c5086d245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:49.570646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e43c03-2535-4cbd-93b7-f85c5086d245 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:49.588574 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 665/1325] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:16:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:16:51.572731 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5145771-56ec-43d0-bdf7-9ef854c9dd45 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:51.573315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5145771-56ec-43d0-bdf7-9ef854c9dd45 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:51.573315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5145771-56ec-43d0-bdf7-9ef854c9dd45 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:51.574932 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 661/1326] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:16:51.580196 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f92339-c04b-4676-956e-73562f4b2f39 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:51.580669 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f92339-c04b-4676-956e-73562f4b2f39 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:51.580880 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f92339-c04b-4676-956e-73562f4b2f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:51.639846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:51.660300 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-01f92339-c04b-4676-956e-73562f4b2f39 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:51.670462 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-01f92339-c04b-4676-956e-73562f4b2f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:51.761488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 666/1327] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:52.046837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5145771-56ec-43d0-bdf7-9ef854c9dd45 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:52.048410 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e5145771-56ec-43d0-bdf7-9ef854c9dd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:52.170851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5145771-56ec-43d0-bdf7-9ef854c9dd45 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:52.251393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8bc40-9f1d-4f3e-b267-9bea076bfe6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:52.251876 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8bc40-9f1d-4f3e-b267-9bea076bfe6f service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:52.252161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8bc40-9f1d-4f3e-b267-9bea076bfe6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:52.252436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8bc40-9f1d-4f3e-b267-9bea076bfe6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:52.254020 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e6f8bc40-9f1d-4f3e-b267-9bea076bfe6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:52.273206 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:52.382584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8bc40-9f1d-4f3e-b267-9bea076bfe6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:52.385101 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8bc40-9f1d-4f3e-b267-9bea076bfe6f service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:52.406474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 662/1328] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:16:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:16:54.411774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76134ef-83b6-422d-98ae-1b4dfbde1b76 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:54.412344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76134ef-83b6-422d-98ae-1b4dfbde1b76 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:54.412344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76134ef-83b6-422d-98ae-1b4dfbde1b76 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:54.413691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 667/1329] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:16:54.419364 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf3940-8ada-435a-b79e-32b157c2c07b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:54.419856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf3940-8ada-435a-b79e-32b157c2c07b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:54.419964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf3940-8ada-435a-b79e-32b157c2c07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:54.491823 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-3fdf3940-8ada-435a-b79e-32b157c2c07b None None] MFA Rules not processed for user `7404abca54674994bf5e6ec8f024692d`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:54.493032 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-3fdf3940-8ada-435a-b79e-32b157c2c07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:54.559332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:54.595215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 663/1330] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:16:54 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:54.880732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76134ef-83b6-422d-98ae-1b4dfbde1b76 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:54.882296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d76134ef-83b6-422d-98ae-1b4dfbde1b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:55.000582 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76134ef-83b6-422d-98ae-1b4dfbde1b76 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:55.022035 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:55.074450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19523869-4862-4453-b042-4ffd5a3257d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:55.075085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19523869-4862-4453-b042-4ffd5a3257d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:55.075363 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19523869-4862-4453-b042-4ffd5a3257d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:55.075631 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19523869-4862-4453-b042-4ffd5a3257d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:55.077238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-19523869-4862-4453-b042-4ffd5a3257d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:55.210224 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19523869-4862-4453-b042-4ffd5a3257d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:55.211714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19523869-4862-4453-b042-4ffd5a3257d2 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:55.232728 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 668/1331] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:16:54 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:16:57.780530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352506a-c283-40e6-a9b9-3e1473c42789 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:57.781105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352506a-c283-40e6-a9b9-3e1473c42789 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:57.781105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352506a-c283-40e6-a9b9-3e1473c42789 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:57.782418 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 664/1332] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:16:57.787823 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a2247-5e9d-4ffd-b402-36f71f66e5a3 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:57.788270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a2247-5e9d-4ffd-b402-36f71f66e5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:57.788365 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a2247-5e9d-4ffd-b402-36f71f66e5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:57.858778 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:57.861672 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-287a2247-5e9d-4ffd-b402-36f71f66e5a3 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:57.862933 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-287a2247-5e9d-4ffd-b402-36f71f66e5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:57.961899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 669/1333] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:58.247881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352506a-c283-40e6-a9b9-3e1473c42789 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:58.249425 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4352506a-c283-40e6-a9b9-3e1473c42789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:58.358113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352506a-c283-40e6-a9b9-3e1473c42789 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:58.431982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167322b6-20f8-43c4-977f-872cb320de8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:58.432594 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167322b6-20f8-43c4-977f-872cb320de8d service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:58.432882 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167322b6-20f8-43c4-977f-872cb320de8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:58.433154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167322b6-20f8-43c4-977f-872cb320de8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:58.434747 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-167322b6-20f8-43c4-977f-872cb320de8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:58.462466 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:58.546601 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167322b6-20f8-43c4-977f-872cb320de8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:58.547972 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167322b6-20f8-43c4-977f-872cb320de8d service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:58.567603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 665/1334] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:16:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:17:00.525506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a1dce-8690-4706-bfbb-5e4d931f38ea None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:00.526049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a1dce-8690-4706-bfbb-5e4d931f38ea None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:00.526049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a1dce-8690-4706-bfbb-5e4d931f38ea None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:00.527340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 670/1335] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:00.532526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e55aca-b2b3-4bf1-8a0f-4215a7caeb80 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:00.532990 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e55aca-b2b3-4bf1-8a0f-4215a7caeb80 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:00.533100 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e55aca-b2b3-4bf1-8a0f-4215a7caeb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:00.613045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-b3e55aca-b2b3-4bf1-8a0f-4215a7caeb80 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:00.614849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b3e55aca-b2b3-4bf1-8a0f-4215a7caeb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:00.693346 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:00.718339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 666/1336] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:01.012673 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a1dce-8690-4706-bfbb-5e4d931f38ea None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:01.014192 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-729a1dce-8690-4706-bfbb-5e4d931f38ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:01.122241 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a1dce-8690-4706-bfbb-5e4d931f38ea None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:01.151760 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:01.196830 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0e39e-f15f-475f-a3b6-ad740ecfc35d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:01.197436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0e39e-f15f-475f-a3b6-ad740ecfc35d service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:01.197714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0e39e-f15f-475f-a3b6-ad740ecfc35d service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:01.197987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0e39e-f15f-475f-a3b6-ad740ecfc35d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:01.199609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-83e0e39e-f15f-475f-a3b6-ad740ecfc35d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:01.309778 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e0e39e-f15f-475f-a3b6-ad740ecfc35d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:01.311182 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e0e39e-f15f-475f-a3b6-ad740ecfc35d service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:01.330097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 671/1337] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:04.149057 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c08c4-55a0-4733-bef1-b836e8c6eea2 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:04.149612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c08c4-55a0-4733-bef1-b836e8c6eea2 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:04.149612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c08c4-55a0-4733-bef1-b836e8c6eea2 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:04.150903 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 667/1338] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:04.156142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06096c17-d9e0-4d7b-bdf1-11133a97c028 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:04.156555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06096c17-d9e0-4d7b-bdf1-11133a97c028 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:04.156692 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06096c17-d9e0-4d7b-bdf1-11133a97c028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:04.223625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-06096c17-d9e0-4d7b-bdf1-11133a97c028 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:04.224862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-06096c17-d9e0-4d7b-bdf1-11133a97c028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:04.233429 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:04.323061 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 672/1339] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:17:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:04.605146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c08c4-55a0-4733-bef1-b836e8c6eea2 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:04.606676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-1a0c08c4-55a0-4733-bef1-b836e8c6eea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:04.712889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c08c4-55a0-4733-bef1-b836e8c6eea2 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:04.785464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee16924-e4e5-49a9-8dd8-8ac7b7857a48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:04.786129 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee16924-e4e5-49a9-8dd8-8ac7b7857a48 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:04.786405 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee16924-e4e5-49a9-8dd8-8ac7b7857a48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:04.786675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee16924-e4e5-49a9-8dd8-8ac7b7857a48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:04.788291 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-2ee16924-e4e5-49a9-8dd8-8ac7b7857a48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:04.824307 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:04.898136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee16924-e4e5-49a9-8dd8-8ac7b7857a48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:04.899510 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee16924-e4e5-49a9-8dd8-8ac7b7857a48 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:04.917594 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 668/1340] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:09.347989 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51d85a-f7d9-4669-90b6-fa196bbcc75b None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:09.348573 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51d85a-f7d9-4669-90b6-fa196bbcc75b None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:09.348838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51d85a-f7d9-4669-90b6-fa196bbcc75b None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:09.350201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 673/1341] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:09.355734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1b29c-0c43-47d4-8de0-ae9a61851b5a None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:09.356171 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1b29c-0c43-47d4-8de0-ae9a61851b5a None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:09.356424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1b29c-0c43-47d4-8de0-ae9a61851b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:09.426494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-bab1b29c-0c43-47d4-8de0-ae9a61851b5a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:09.427795 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-bab1b29c-0c43-47d4-8de0-ae9a61851b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:09.518475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:09.530340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 669/1342] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:09.813895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51d85a-f7d9-4669-90b6-fa196bbcc75b None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:09.815437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7a51d85a-f7d9-4669-90b6-fa196bbcc75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:09.930920 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51d85a-f7d9-4669-90b6-fa196bbcc75b None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:09.974289 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:10.011447 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50f314-7f52-460e-90d2-5b6a88608ef0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:10.012099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50f314-7f52-460e-90d2-5b6a88608ef0 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:10.012381 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50f314-7f52-460e-90d2-5b6a88608ef0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:10.012655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50f314-7f52-460e-90d2-5b6a88608ef0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:10.014366 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-cf50f314-7f52-460e-90d2-5b6a88608ef0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:10.129699 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50f314-7f52-460e-90d2-5b6a88608ef0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:10.131456 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50f314-7f52-460e-90d2-5b6a88608ef0 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:10.151049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 674/1343] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:13.677333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9697357-9b68-49a6-98ca-3546e93da5be None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:13.677895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9697357-9b68-49a6-98ca-3546e93da5be None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:13.677895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9697357-9b68-49a6-98ca-3546e93da5be None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:13.679524 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 670/1344] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:17:13.685553 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38022229-2990-4e44-8b6e-1b8440080ed7 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:13.686097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38022229-2990-4e44-8b6e-1b8440080ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:13.686097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38022229-2990-4e44-8b6e-1b8440080ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:13.759788 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-38022229-2990-4e44-8b6e-1b8440080ed7 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:13.760985 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-38022229-2990-4e44-8b6e-1b8440080ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:13.779192 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:13.862303 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 675/1345] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:17:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:14.148204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9697357-9b68-49a6-98ca-3546e93da5be None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:14.149811 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b9697357-9b68-49a6-98ca-3546e93da5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:14.256659 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9697357-9b68-49a6-98ca-3546e93da5be None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:14.335999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d12d3-0401-4a1c-b6ff-ea8f8a4dd652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:14.336661 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d12d3-0401-4a1c-b6ff-ea8f8a4dd652 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:14.336937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d12d3-0401-4a1c-b6ff-ea8f8a4dd652 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:14.337202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d12d3-0401-4a1c-b6ff-ea8f8a4dd652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:14.338795 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f82d12d3-0401-4a1c-b6ff-ea8f8a4dd652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:14.383939 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:14.480436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d12d3-0401-4a1c-b6ff-ea8f8a4dd652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:14.482297 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d12d3-0401-4a1c-b6ff-ea8f8a4dd652 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:14.505503 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 671/1346] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:19.215717 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2c9be-3583-4d02-b975-e2232cdf544d None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:19.216275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2c9be-3583-4d02-b975-e2232cdf544d None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:19.216275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2c9be-3583-4d02-b975-e2232cdf544d None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:19.217641 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 676/1347] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:19.223436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07765099-13f7-4af7-992c-0ce1e7dd9ae4 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:19.223923 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07765099-13f7-4af7-992c-0ce1e7dd9ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:19.224032 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07765099-13f7-4af7-992c-0ce1e7dd9ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:19.301457 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-07765099-13f7-4af7-992c-0ce1e7dd9ae4 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:19.302722 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-07765099-13f7-4af7-992c-0ce1e7dd9ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:19.424403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:19.428417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 672/1348] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:19.720051 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2c9be-3583-4d02-b975-e2232cdf544d None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:19.721884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c3d2c9be-3583-4d02-b975-e2232cdf544d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:19.861808 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2c9be-3583-4d02-b975-e2232cdf544d None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:19.918619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:19.947150 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5f89f-daba-4fbc-8586-ec6d00ee5ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:19.947773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5f89f-daba-4fbc-8586-ec6d00ee5ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:19.948052 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5f89f-daba-4fbc-8586-ec6d00ee5ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:19.958833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5f89f-daba-4fbc-8586-ec6d00ee5ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:19.958833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e8c5f89f-daba-4fbc-8586-ec6d00ee5ca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:20.061256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c5f89f-daba-4fbc-8586-ec6d00ee5ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:20.062603 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c5f89f-daba-4fbc-8586-ec6d00ee5ca8 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:20.080572 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 677/1349] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:25.129803 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea9cef-3340-4cf7-80a4-103c979bbf68 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:25.130669 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea9cef-3340-4cf7-80a4-103c979bbf68 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:25.131121 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea9cef-3340-4cf7-80a4-103c979bbf68 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:25.136466 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 673/1350] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 31 19:17:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:25.141357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbaf02b-e050-4deb-9aa8-4f0b02422879 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:25.141796 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbaf02b-e050-4deb-9aa8-4f0b02422879 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:25.141909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbaf02b-e050-4deb-9aa8-4f0b02422879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:25.245318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-cdbaf02b-e050-4deb-9aa8-4f0b02422879 None None] MFA Rules not processed for user `eced13cc4030423abe63bb1527f6d35a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:25.246989 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-cdbaf02b-e050-4deb-9aa8-4f0b02422879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:25.270979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:25.398311 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 678/1351] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 31 19:17:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:29.728373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc144bb-c8b6-4e9a-b7bc-42857bdbdd4f None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.728951 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc144bb-c8b6-4e9a-b7bc-42857bdbdd4f None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.728951 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc144bb-c8b6-4e9a-b7bc-42857bdbdd4f None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.730551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 674/1352] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 31 19:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:29.736662 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c522d-a73c-4bd0-ab57-b7d6719e53a9 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.737567 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c522d-a73c-4bd0-ab57-b7d6719e53a9 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.738377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c522d-a73c-4bd0-ab57-b7d6719e53a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.821619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-8e7c522d-a73c-4bd0-ab57-b7d6719e53a9 None None] MFA Rules not processed for user `eced13cc4030423abe63bb1527f6d35a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:29.822843 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8e7c522d-a73c-4bd0-ab57-b7d6719e53a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:29.902171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:29.940574 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 679/1353] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 31 19:17:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:31.304912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b6247-8568-46be-a744-1f079e68ab6c None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:31.305451 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b6247-8568-46be-a744-1f079e68ab6c None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:31.305451 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b6247-8568-46be-a744-1f079e68ab6c None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:31.306797 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 675/1354] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 31 19:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:31.312081 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757a7c7-b336-42c4-aeda-3304719c88f9 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:31.312483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757a7c7-b336-42c4-aeda-3304719c88f9 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:31.312765 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757a7c7-b336-42c4-aeda-3304719c88f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:31.392799 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a757a7c7-b336-42c4-aeda-3304719c88f9 None None] MFA Rules not processed for user `eced13cc4030423abe63bb1527f6d35a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:31.394209 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a757a7c7-b336-42c4-aeda-3304719c88f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:31.510314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 680/1355] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 31 19:17:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:35.068819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b5953-f153-4c9c-a0f6-4c2a46bfefa6 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:35.069377 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b5953-f153-4c9c-a0f6-4c2a46bfefa6 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:35.069377 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b5953-f153-4c9c-a0f6-4c2a46bfefa6 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:35.070898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 676/1356] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:35.076320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4783a69-929d-401c-bc30-dd2ae06b9c05 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:35.076765 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4783a69-929d-401c-bc30-dd2ae06b9c05 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:35.076996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4783a69-929d-401c-bc30-dd2ae06b9c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:35.117599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:35.152010 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d4783a69-929d-401c-bc30-dd2ae06b9c05 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:35.153670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d4783a69-929d-401c-bc30-dd2ae06b9c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:35.277321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 681/1357] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:17:35.639835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f88f9d-e730-4146-8a47-a04623019e85 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:35.640092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f88f9d-e730-4146-8a47-a04623019e85 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:35.640741 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f88f9d-e730-4146-8a47-a04623019e85 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:35.642256 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 677/1358] 10.222.0.59 () {54 vars in 962 bytes} [Wed Aug 31 19:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:35.656084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f742e4d-d889-4210-bdec-9d150ecf6c9b None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:35.656506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f742e4d-d889-4210-bdec-9d150ecf6c9b None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:35.658829 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f742e4d-d889-4210-bdec-9d150ecf6c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:35.755023 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-9f742e4d-d889-4210-bdec-9d150ecf6c9b None None] MFA Rules not processed for user `d461422ce98442bcbe6b254a8dad5e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:35.756562 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9f742e4d-d889-4210-bdec-9d150ecf6c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:35.820595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:35.910950 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 682/1359] 10.222.0.59 () {58 vars in 1059 bytes} [Wed Aug 31 19:17:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:35.920267 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f88f9d-e730-4146-8a47-a04623019e85 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:35.922654 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-75f88f9d-e730-4146-8a47-a04623019e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:36.090108 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f88f9d-e730-4146-8a47-a04623019e85 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:36.172124 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:36.221383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f93656-43ad-4efa-af60-48c5e44223b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:36.223404 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f93656-43ad-4efa-af60-48c5e44223b6 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:36.223878 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f93656-43ad-4efa-af60-48c5e44223b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:36.224310 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f93656-43ad-4efa-af60-48c5e44223b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:36.226472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-70f93656-43ad-4efa-af60-48c5e44223b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:36.332904 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f93656-43ad-4efa-af60-48c5e44223b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:36.334631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f93656-43ad-4efa-af60-48c5e44223b6 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:36.355106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 678/1360] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:38.553976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8acee6-a071-4de1-b411-75e5368d34bc None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:38.554490 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8acee6-a071-4de1-b411-75e5368d34bc None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:38.554663 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8acee6-a071-4de1-b411-75e5368d34bc None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:38.556195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 683/1361] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:38.562818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38541fce-841e-4a99-bffa-1249d5462d09 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:38.563309 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38541fce-841e-4a99-bffa-1249d5462d09 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:38.563594 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38541fce-841e-4a99-bffa-1249d5462d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:38.643700 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-38541fce-841e-4a99-bffa-1249d5462d09 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:38.646051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-38541fce-841e-4a99-bffa-1249d5462d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:38.688898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:38.766906 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 679/1362] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:39.067609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8acee6-a071-4de1-b411-75e5368d34bc None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:39.069201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4a8acee6-a071-4de1-b411-75e5368d34bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:39.231135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8acee6-a071-4de1-b411-75e5368d34bc None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:39.242981 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:39.369386 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53eea9a-544b-49bd-b229-8cd60e40a4d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:39.370009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53eea9a-544b-49bd-b229-8cd60e40a4d8 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:39.370291 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53eea9a-544b-49bd-b229-8cd60e40a4d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:39.370561 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53eea9a-544b-49bd-b229-8cd60e40a4d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:39.372215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d53eea9a-544b-49bd-b229-8cd60e40a4d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:39.486507 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53eea9a-544b-49bd-b229-8cd60e40a4d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:39.488406 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53eea9a-544b-49bd-b229-8cd60e40a4d8 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:39.510703 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 684/1363] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:41.963898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1c6ff-c1db-4592-a2ff-ae8597629545 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:41.964465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1c6ff-c1db-4592-a2ff-ae8597629545 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:41.964553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1c6ff-c1db-4592-a2ff-ae8597629545 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:41.966010 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 680/1364] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:41.971476 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08307d8b-0d1a-445a-85a0-7540aeeee625 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:41.972499 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08307d8b-0d1a-445a-85a0-7540aeeee625 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:41.973020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08307d8b-0d1a-445a-85a0-7540aeeee625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:41.990481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:42.052594 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-08307d8b-0d1a-445a-85a0-7540aeeee625 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:42.053828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-08307d8b-0d1a-445a-85a0-7540aeeee625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:42.175389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 685/1365] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:42.491604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5f06e-e6d4-48ca-ad07-239219642132 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:42.491926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5f06e-e6d4-48ca-ad07-239219642132 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:42.492157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5f06e-e6d4-48ca-ad07-239219642132 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:42.493755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 681/1366] 10.222.0.59 () {54 vars in 962 bytes} [Wed Aug 31 19:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:42.499618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444fc6b-ef1e-4629-81f8-4934708fbcf3 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:42.499979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444fc6b-ef1e-4629-81f8-4934708fbcf3 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:42.500334 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444fc6b-ef1e-4629-81f8-4934708fbcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:42.590020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-0444fc6b-ef1e-4629-81f8-4934708fbcf3 None None] MFA Rules not processed for user `d461422ce98442bcbe6b254a8dad5e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:42.591257 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-0444fc6b-ef1e-4629-81f8-4934708fbcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:42.633761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:42.735931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 686/1367] 10.222.0.59 () {58 vars in 1059 bytes} [Wed Aug 31 19:17:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:42.745402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5f06e-e6d4-48ca-ad07-239219642132 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:42.746965 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-0df5f06e-e6d4-48ca-ad07-239219642132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:42.957382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5f06e-e6d4-48ca-ad07-239219642132 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:42.987131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:43.100380 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203db518-8a73-4e55-9e1d-f39416487445 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:43.101054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203db518-8a73-4e55-9e1d-f39416487445 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:43.101341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203db518-8a73-4e55-9e1d-f39416487445 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:43.101644 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203db518-8a73-4e55-9e1d-f39416487445 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:43.103371 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-203db518-8a73-4e55-9e1d-f39416487445 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:43.211003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203db518-8a73-4e55-9e1d-f39416487445 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:43.212584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203db518-8a73-4e55-9e1d-f39416487445 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:43.234047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 682/1368] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:43.601872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df360c-9083-4343-822f-66e33d21a1b4 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:43.602515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df360c-9083-4343-822f-66e33d21a1b4 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:43.602515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df360c-9083-4343-822f-66e33d21a1b4 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:43.603894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 687/1369] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:43.610186 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeeebf-fcdd-4ace-ad2f-5946ab4cffde None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:43.610656 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeeebf-fcdd-4ace-ad2f-5946ab4cffde None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:43.611069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeeebf-fcdd-4ace-ad2f-5946ab4cffde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:43.652060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:43.697021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-2caeeebf-fcdd-4ace-ad2f-5946ab4cffde None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:43.698593 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-2caeeebf-fcdd-4ace-ad2f-5946ab4cffde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:43.824851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 683/1370] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:44.438512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e30e48-35d5-4c58-9314-6f06361bc442 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:44.438699 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e30e48-35d5-4c58-9314-6f06361bc442 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:44.439053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e30e48-35d5-4c58-9314-6f06361bc442 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:44.440684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 688/1371] 10.222.0.59 () {54 vars in 964 bytes} [Wed Aug 31 19:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:17:44.445780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e581e-92d0-4a15-9fef-b15f1d3a39e9 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:44.446246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e581e-92d0-4a15-9fef-b15f1d3a39e9 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:44.446444 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e581e-92d0-4a15-9fef-b15f1d3a39e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:44.556560 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-335e581e-92d0-4a15-9fef-b15f1d3a39e9 None None] MFA Rules not processed for user `108a698aa3e6477b9ae896d68f22bd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:44.557986 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-335e581e-92d0-4a15-9fef-b15f1d3a39e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:44.616454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:44.652395 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 684/1372] 10.222.0.59 () {58 vars in 1061 bytes} [Wed Aug 31 19:17:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:44.660927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e30e48-35d5-4c58-9314-6f06361bc442 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:44.663120 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-84e30e48-35d5-4c58-9314-6f06361bc442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:44.790949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e30e48-35d5-4c58-9314-6f06361bc442 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:44.836794 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:44.873688 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a419b0-efe6-4721-a2c8-97a337177ca7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:44.874308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a419b0-efe6-4721-a2c8-97a337177ca7 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:44.874590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a419b0-efe6-4721-a2c8-97a337177ca7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:44.874864 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a419b0-efe6-4721-a2c8-97a337177ca7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:44.876495 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-b4a419b0-efe6-4721-a2c8-97a337177ca7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:45.000450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a419b0-efe6-4721-a2c8-97a337177ca7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:45.001995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a419b0-efe6-4721-a2c8-97a337177ca7 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:45.027172 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 689/1373] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:45.500211 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b7156-8af8-4460-8d0e-86e3d3a2f5d8 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:45.500770 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b7156-8af8-4460-8d0e-86e3d3a2f5d8 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:45.500770 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b7156-8af8-4460-8d0e-86e3d3a2f5d8 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:45.502098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 685/1374] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:45.507830 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c5d7c-73fc-4a09-a0ba-e6b980670d8a None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:45.508527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c5d7c-73fc-4a09-a0ba-e6b980670d8a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:45.509521 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c5d7c-73fc-4a09-a0ba-e6b980670d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:45.645439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-7b5c5d7c-73fc-4a09-a0ba-e6b980670d8a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:45.645439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7b5c5d7c-73fc-4a09-a0ba-e6b980670d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:45.645439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:45.696310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 690/1375] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:45.988850 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b7156-8af8-4460-8d0e-86e3d3a2f5d8 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:45.990393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-0d1b7156-8af8-4460-8d0e-86e3d3a2f5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:46.148992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b7156-8af8-4460-8d0e-86e3d3a2f5d8 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:46.205088 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:46.268837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfbfd3-6e2f-48de-925c-b64d0cd05cb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:46.269693 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfbfd3-6e2f-48de-925c-b64d0cd05cb1 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:46.269978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfbfd3-6e2f-48de-925c-b64d0cd05cb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:46.270344 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfbfd3-6e2f-48de-925c-b64d0cd05cb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:46.272018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-94bfbfd3-6e2f-48de-925c-b64d0cd05cb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:46.369235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfbfd3-6e2f-48de-925c-b64d0cd05cb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:46.370826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfbfd3-6e2f-48de-925c-b64d0cd05cb1 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:46.389421 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 686/1376] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:48.476475 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130a5c6-274a-4286-acea-bcbbea3eae57 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:48.477059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130a5c6-274a-4286-acea-bcbbea3eae57 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:48.477059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130a5c6-274a-4286-acea-bcbbea3eae57 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:48.478514 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 691/1377] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:48.484379 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66638302-f046-4e91-bd5b-3971c4797c14 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:48.484817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66638302-f046-4e91-bd5b-3971c4797c14 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:48.484930 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66638302-f046-4e91-bd5b-3971c4797c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:48.560762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-66638302-f046-4e91-bd5b-3971c4797c14 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:48.562244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-66638302-f046-4e91-bd5b-3971c4797c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:48.571765 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:48.682990 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 687/1378] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:48.992555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130a5c6-274a-4286-acea-bcbbea3eae57 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:48.994053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-1130a5c6-274a-4286-acea-bcbbea3eae57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:49.174256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130a5c6-274a-4286-acea-bcbbea3eae57 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:49.179567 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:49.313212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97b377-464a-48f4-9a2e-0097521e0205 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:49.316205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97b377-464a-48f4-9a2e-0097521e0205 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:49.316205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97b377-464a-48f4-9a2e-0097521e0205 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:49.316205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97b377-464a-48f4-9a2e-0097521e0205 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:49.317015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-eb97b377-464a-48f4-9a2e-0097521e0205 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:49.419896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97b377-464a-48f4-9a2e-0097521e0205 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:49.421673 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97b377-464a-48f4-9a2e-0097521e0205 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:49.443200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 692/1379] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:51.850241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677371e-55fe-465c-a3dc-9a06dddba4a7 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:51.850823 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677371e-55fe-465c-a3dc-9a06dddba4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:51.850823 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677371e-55fe-465c-a3dc-9a06dddba4a7 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:51.852297 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 688/1380] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:51.857759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b2b97-0877-48b1-ab97-46ef0d8b954b None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:51.858229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b2b97-0877-48b1-ab97-46ef0d8b954b None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:51.858336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b2b97-0877-48b1-ab97-46ef0d8b954b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:51.870152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:51.929726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-426b2b97-0877-48b1-ab97-46ef0d8b954b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:51.931187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-426b2b97-0877-48b1-ab97-46ef0d8b954b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:52.050916 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 693/1381] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:52.340149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677371e-55fe-465c-a3dc-9a06dddba4a7 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:52.341836 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-1677371e-55fe-465c-a3dc-9a06dddba4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:52.497239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:52.508720 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677371e-55fe-465c-a3dc-9a06dddba4a7 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:52.641373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274df488-b78e-4b7f-97b6-7e6f9a15f1f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:52.642017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274df488-b78e-4b7f-97b6-7e6f9a15f1f2 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:52.642299 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274df488-b78e-4b7f-97b6-7e6f9a15f1f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:52.642569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274df488-b78e-4b7f-97b6-7e6f9a15f1f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:52.644825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-274df488-b78e-4b7f-97b6-7e6f9a15f1f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:52.745928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274df488-b78e-4b7f-97b6-7e6f9a15f1f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:52.747838 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274df488-b78e-4b7f-97b6-7e6f9a15f1f2 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:52.759946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:52.769527 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 689/1382] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:54.940850 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b845233-6ccf-4f66-aee8-d7d7586b8f90 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:54.942029 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b845233-6ccf-4f66-aee8-d7d7586b8f90 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:54.942029 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b845233-6ccf-4f66-aee8-d7d7586b8f90 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:54.943026 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 694/1383] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:17:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:17:54.949608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39961357-1be4-499b-a3e4-72354bf7ca87 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:54.950070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39961357-1be4-499b-a3e4-72354bf7ca87 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:54.950371 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39961357-1be4-499b-a3e4-72354bf7ca87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:55.032941 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-39961357-1be4-499b-a3e4-72354bf7ca87 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:55.034333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-39961357-1be4-499b-a3e4-72354bf7ca87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:55.136344 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 690/1384] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:55.433810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b845233-6ccf-4f66-aee8-d7d7586b8f90 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:55.435341 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7b845233-6ccf-4f66-aee8-d7d7586b8f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:55.542466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:55.612022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b845233-6ccf-4f66-aee8-d7d7586b8f90 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:55.739961 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc192521-f733-4671-8d7c-aa128c98087f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:55.740581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc192521-f733-4671-8d7c-aa128c98087f service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:55.740858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc192521-f733-4671-8d7c-aa128c98087f service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:55.741127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc192521-f733-4671-8d7c-aa128c98087f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:55.742713 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-cc192521-f733-4671-8d7c-aa128c98087f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:55.801897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:55.844869 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc192521-f733-4671-8d7c-aa128c98087f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:55.847289 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc192521-f733-4671-8d7c-aa128c98087f service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:55.866321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 695/1385] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:57.838502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9faa86-a402-46c7-8949-dc7dc4267c40 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:57.839237 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9faa86-a402-46c7-8949-dc7dc4267c40 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:57.839237 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9faa86-a402-46c7-8949-dc7dc4267c40 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:57.840420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 691/1386] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:57.847128 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ef330-a673-4a47-b5d3-6d6891175076 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:57.847688 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ef330-a673-4a47-b5d3-6d6891175076 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:57.847862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ef330-a673-4a47-b5d3-6d6891175076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:57.923312 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-ca7ef330-a673-4a47-b5d3-6d6891175076 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:57.925427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ca7ef330-a673-4a47-b5d3-6d6891175076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:58.039302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 696/1387] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:58.368436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9faa86-a402-46c7-8949-dc7dc4267c40 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:58.369928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-fd9faa86-a402-46c7-8949-dc7dc4267c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:58.456357 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:58.530970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9faa86-a402-46c7-8949-dc7dc4267c40 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:58.648490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368a733-daf0-4fd4-b91c-e80762a77736 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:58.649110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368a733-daf0-4fd4-b91c-e80762a77736 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:58.649394 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368a733-daf0-4fd4-b91c-e80762a77736 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:58.649670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368a733-daf0-4fd4-b91c-e80762a77736 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:58.651326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-7368a733-daf0-4fd4-b91c-e80762a77736 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:58.694593 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:58.752200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368a733-daf0-4fd4-b91c-e80762a77736 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:58.754670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368a733-daf0-4fd4-b91c-e80762a77736 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:58.774480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 692/1388] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:00.885372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6204745-7dc9-4723-a2cb-485615995ae1 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:00.885932 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6204745-7dc9-4723-a2cb-485615995ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:00.886050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6204745-7dc9-4723-a2cb-485615995ae1 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:00.887490 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 697/1389] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:00.893237 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1702121-47a3-4aec-9300-cd7e95d8eec7 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:00.893904 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1702121-47a3-4aec-9300-cd7e95d8eec7 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:00.894119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1702121-47a3-4aec-9300-cd7e95d8eec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:00.920432 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae36ea-7207-4faf-a2d5-4f4f1300f9d1 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:00.920690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae36ea-7207-4faf-a2d5-4f4f1300f9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:00.920962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae36ea-7207-4faf-a2d5-4f4f1300f9d1 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:00.922333 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 698/1390] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:18:00.928813 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3cccc-5661-4eb1-9e02-73208712aca3 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:00.929102 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3cccc-5661-4eb1-9e02-73208712aca3 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:00.929384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3cccc-5661-4eb1-9e02-73208712aca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:00.967081 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:00.983960 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-e1702121-47a3-4aec-9300-cd7e95d8eec7 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:00.985316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e1702121-47a3-4aec-9300-cd7e95d8eec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:01.006032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-63f3cccc-5661-4eb1-9e02-73208712aca3 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:01.007304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-63f3cccc-5661-4eb1-9e02-73208712aca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:01.107362 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 693/1391] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:01.118923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 699/1392] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:18:01.418806 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1702121-47a3-4aec-9300-cd7e95d8eec7 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:01.420526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e1702121-47a3-4aec-9300-cd7e95d8eec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:01.437582 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b47b73-1aea-43b8-a2e1-949229d8914a None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:01.438113 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b47b73-1aea-43b8-a2e1-949229d8914a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:01.438288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b47b73-1aea-43b8-a2e1-949229d8914a None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:01.439117 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:01.441254 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 700/1393] 10.222.0.59 () {54 vars in 967 bytes} [Wed Aug 31 19:18:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:18:01.445862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5444e99-6f15-4690-a1a5-8b31335c1a31 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:01.446353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5444e99-6f15-4690-a1a5-8b31335c1a31 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:01.447080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5444e99-6f15-4690-a1a5-8b31335c1a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:01.540047 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a5444e99-6f15-4690-a1a5-8b31335c1a31 None None] MFA Rules not processed for user `f281d778f1404984b4d778a1c26aef1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:01.541187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a5444e99-6f15-4690-a1a5-8b31335c1a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:01.556594 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1702121-47a3-4aec-9300-cd7e95d8eec7 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:01.595412 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:01.624134 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 701/1394] 10.222.0.59 () {58 vars in 1064 bytes} [Wed Aug 31 19:18:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:18:01.632312 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5444e99-6f15-4690-a1a5-8b31335c1a31 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:01.642512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5444e99-6f15-4690-a1a5-8b31335c1a31 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:01.685581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643d7f1-ac7a-4b05-a478-35fe484f6545 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:01.686303 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643d7f1-ac7a-4b05-a478-35fe484f6545 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:01.686581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643d7f1-ac7a-4b05-a478-35fe484f6545 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:01.686850 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643d7f1-ac7a-4b05-a478-35fe484f6545 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:01.688543 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-6643d7f1-ac7a-4b05-a478-35fe484f6545 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:01.707426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:01.719438 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230c6b3-9b19-4809-993b-37e2a9fa5364 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f281d778f1404984b4d778a1c26aef1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:01.720053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230c6b3-9b19-4809-993b-37e2a9fa5364 service glance] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:01.720335 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230c6b3-9b19-4809-993b-37e2a9fa5364 service glance] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:01.720609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230c6b3-9b19-4809-993b-37e2a9fa5364 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:01.722888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7230c6b3-9b19-4809-993b-37e2a9fa5364 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:01.795399 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643d7f1-ac7a-4b05-a478-35fe484f6545 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:01.795399 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643d7f1-ac7a-4b05-a478-35fe484f6545 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:01.814569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 694/1395] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:01.842377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230c6b3-9b19-4809-993b-37e2a9fa5364 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:01.843859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230c6b3-9b19-4809-993b-37e2a9fa5364 service glance] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:01.859024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:01.863984 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 702/1396] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:18:02.791699 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302dc39-ab08-4a81-8a3a-fa5c5fee04ea None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:02.792176 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302dc39-ab08-4a81-8a3a-fa5c5fee04ea None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:02.792258 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302dc39-ab08-4a81-8a3a-fa5c5fee04ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:02.864365 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-f302dc39-ab08-4a81-8a3a-fa5c5fee04ea None None] MFA Rules not processed for user `1ee990b1658e49cc8bc2fe15e7dc36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:02.865896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f302dc39-ab08-4a81-8a3a-fa5c5fee04ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:02.953204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:02.957466 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 695/1397] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 31 19:18:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:02.999632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d0459-f014-4807-83ff-55db9783d618 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:03.000112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d0459-f014-4807-83ff-55db9783d618 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:03.000217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d0459-f014-4807-83ff-55db9783d618 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:03.001577 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 703/1398] 10.222.0.59 () {54 vars in 943 bytes} [Wed Aug 31 19:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:03.008428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18a56a-4aa6-48f6-877e-58b5096d1d6d None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:03.008723 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18a56a-4aa6-48f6-877e-58b5096d1d6d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:03.009002 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18a56a-4aa6-48f6-877e-58b5096d1d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:03.089706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-2f18a56a-4aa6-48f6-877e-58b5096d1d6d None None] MFA Rules not processed for user `25ed58369ffd46e8b7fa1880662926e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:03.090992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-2f18a56a-4aa6-48f6-877e-58b5096d1d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:03.203192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 696/1399] 10.222.0.59 () {58 vars in 1040 bytes} [Wed Aug 31 19:18:03 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:03.213422 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d0459-f014-4807-83ff-55db9783d618 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:03.215105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-005d0459-f014-4807-83ff-55db9783d618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:03.364447 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d0459-f014-4807-83ff-55db9783d618 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:03.453076 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:03.480301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701513b-ae9f-498b-b977-f610c7d17c97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25ed58369ffd46e8b7fa1880662926e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:03.480301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701513b-ae9f-498b-b977-f610c7d17c97 service swift] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:03.480301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701513b-ae9f-498b-b977-f610c7d17c97 service swift] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:03.480301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701513b-ae9f-498b-b977-f610c7d17c97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:03.480838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9701513b-ae9f-498b-b977-f610c7d17c97 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:03.563098 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701513b-ae9f-498b-b977-f610c7d17c97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:03.564702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701513b-ae9f-498b-b977-f610c7d17c97 service swift] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:03.573726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 704/1400] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 31 19:18:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:18:04.005035 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53283ec-ef34-4e1d-bcbc-a0ef9802c96b None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:04.005035 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53283ec-ef34-4e1d-bcbc-a0ef9802c96b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:04.005035 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53283ec-ef34-4e1d-bcbc-a0ef9802c96b None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:04.006230 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 697/1401] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:04.011791 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639b9cc-810e-4c65-ac93-4bc7bd88ebb1 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:04.012278 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639b9cc-810e-4c65-ac93-4bc7bd88ebb1 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:04.012384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639b9cc-810e-4c65-ac93-4bc7bd88ebb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:04.087347 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-4639b9cc-810e-4c65-ac93-4bc7bd88ebb1 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:04.088873 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4639b9cc-810e-4c65-ac93-4bc7bd88ebb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:04.162444 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:04.193678 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 705/1402] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:04.489168 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53283ec-ef34-4e1d-bcbc-a0ef9802c96b None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:04.490690 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e53283ec-ef34-4e1d-bcbc-a0ef9802c96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:04.566721 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:04.665995 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53283ec-ef34-4e1d-bcbc-a0ef9802c96b None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:04.800737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fc80e-b4d7-4586-814a-0f61de7f6627 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:04.800737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fc80e-b4d7-4586-814a-0f61de7f6627 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:04.802528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fc80e-b4d7-4586-814a-0f61de7f6627 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:04.802839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fc80e-b4d7-4586-814a-0f61de7f6627 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:04.804539 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b23fc80e-b4d7-4586-814a-0f61de7f6627 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:04.827600 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:04.910408 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fc80e-b4d7-4586-814a-0f61de7f6627 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:04.912094 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fc80e-b4d7-4586-814a-0f61de7f6627 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:04.931383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 698/1403] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:07.087085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c83bb9-8fd2-4d9c-94bd-8ba2597105bf None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:07.172514 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c83bb9-8fd2-4d9c-94bd-8ba2597105bf None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:07.172514 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c83bb9-8fd2-4d9c-94bd-8ba2597105bf None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:07.172514 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 706/1404] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:07.172863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9bf26f-db63-495d-a7e3-8b8450a5fea8 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:07.172863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9bf26f-db63-495d-a7e3-8b8450a5fea8 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:07.172863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9bf26f-db63-495d-a7e3-8b8450a5fea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:07.195212 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-4c9bf26f-db63-495d-a7e3-8b8450a5fea8 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:07.196817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4c9bf26f-db63-495d-a7e3-8b8450a5fea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:07.335991 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:07.346939 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 699/1405] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:07.640902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c83bb9-8fd2-4d9c-94bd-8ba2597105bf None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:07.640902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-23c83bb9-8fd2-4d9c-94bd-8ba2597105bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:07.825364 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c83bb9-8fd2-4d9c-94bd-8ba2597105bf None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:07.888712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:07.976661 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ed422-df23-4891-bea9-ecf437c9ffdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:07.977202 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ed422-df23-4891-bea9-ecf437c9ffdc service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:07.977483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ed422-df23-4891-bea9-ecf437c9ffdc service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:07.977754 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ed422-df23-4891-bea9-ecf437c9ffdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:07.979418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e12ed422-df23-4891-bea9-ecf437c9ffdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:08.091336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ed422-df23-4891-bea9-ecf437c9ffdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:08.093900 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ed422-df23-4891-bea9-ecf437c9ffdc service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:08.117799 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 707/1406] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:10.528506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449ee85-d38c-4c71-add8-5f4e136ae0bf None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:10.529045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449ee85-d38c-4c71-add8-5f4e136ae0bf None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:10.529045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449ee85-d38c-4c71-add8-5f4e136ae0bf None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:10.530495 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 700/1407] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:10.536140 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c4a8c-2ad6-48af-9c10-238154850131 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:10.536643 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c4a8c-2ad6-48af-9c10-238154850131 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:10.536746 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c4a8c-2ad6-48af-9c10-238154850131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:10.610757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-530c4a8c-2ad6-48af-9c10-238154850131 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:10.612189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-530c4a8c-2ad6-48af-9c10-238154850131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:10.628472 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:10.727892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 708/1408] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:11.014929 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449ee85-d38c-4c71-add8-5f4e136ae0bf None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:11.016527 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b449ee85-d38c-4c71-add8-5f4e136ae0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:11.198883 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449ee85-d38c-4c71-add8-5f4e136ae0bf None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:11.291480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:11.348419 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ddbd2-f9ff-4a7c-a2b1-02d6ff1ca217 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:11.349300 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ddbd2-f9ff-4a7c-a2b1-02d6ff1ca217 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:11.349649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ddbd2-f9ff-4a7c-a2b1-02d6ff1ca217 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:11.349945 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ddbd2-f9ff-4a7c-a2b1-02d6ff1ca217 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:11.351817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-741ddbd2-f9ff-4a7c-a2b1-02d6ff1ca217 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:11.466791 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ddbd2-f9ff-4a7c-a2b1-02d6ff1ca217 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:11.469198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ddbd2-f9ff-4a7c-a2b1-02d6ff1ca217 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:11.504275 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 701/1409] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:20.719506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=38 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:21.486646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=64 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:28.187059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2421d5-cccb-40e4-80f2-ba3752abfccc None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:28.187652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2421d5-cccb-40e4-80f2-ba3752abfccc None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:28.187652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2421d5-cccb-40e4-80f2-ba3752abfccc None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:28.188952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 709/1410] 10.222.0.59 () {54 vars in 918 bytes} [Wed Aug 31 19:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:28.195003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233ef06-6b36-463f-b3a6-39f8e8084c66 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:28.195439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233ef06-6b36-463f-b3a6-39f8e8084c66 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:28.195573 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233ef06-6b36-463f-b3a6-39f8e8084c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:28.271124 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-6233ef06-6b36-463f-b3a6-39f8e8084c66 None None] MFA Rules not processed for user `eced13cc4030423abe63bb1527f6d35a`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:28.272365 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-6233ef06-6b36-463f-b3a6-39f8e8084c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:28.382934 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 702/1411] 10.222.0.59 () {58 vars in 1015 bytes} [Wed Aug 31 19:18:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:28.397519 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15d5f0-4b82-46dc-aeb6-20d6819f2b1d None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:28.397814 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15d5f0-4b82-46dc-aeb6-20d6819f2b1d None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:28.423195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15d5f0-4b82-46dc-aeb6-20d6819f2b1d None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:28.423195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 710/1412] 10.222.0.59 () {54 vars in 918 bytes} [Wed Aug 31 19:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:28.423480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b9b78-07a8-4864-9845-5c9e3f49e64c None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:28.423480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b9b78-07a8-4864-9845-5c9e3f49e64c None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:28.423480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b9b78-07a8-4864-9845-5c9e3f49e64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:28.481601 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:28.483485 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-439b9b78-07a8-4864-9845-5c9e3f49e64c None None] MFA Rules not processed for user `d461422ce98442bcbe6b254a8dad5e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:28.484718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-439b9b78-07a8-4864-9845-5c9e3f49e64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:28.604766 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 703/1413] 10.222.0.59 () {58 vars in 1015 bytes} [Wed Aug 31 19:18:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:31.010216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b342bac-9659-409c-980a-5675eb81a907 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:31.010755 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b342bac-9659-409c-980a-5675eb81a907 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:31.010755 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b342bac-9659-409c-980a-5675eb81a907 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:31.012141 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:31.018188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af84e5-be8b-450e-849f-529820bb1c1c None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:31.018668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af84e5-be8b-450e-849f-529820bb1c1c None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:31.018908 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af84e5-be8b-450e-849f-529820bb1c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:31.088191 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-a6af84e5-be8b-450e-849f-529820bb1c1c None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:31.089438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a6af84e5-be8b-450e-849f-529820bb1c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:31.145811 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:31.189870 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:18:31.479269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b342bac-9659-409c-980a-5675eb81a907 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:31.480823 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4b342bac-9659-409c-980a-5675eb81a907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:31.633107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b342bac-9659-409c-980a-5675eb81a907 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:31.708775 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:31.750601 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb096a-be08-4f20-bd83-cfdcdb3ae1fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:31.751240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb096a-be08-4f20-bd83-cfdcdb3ae1fa service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:31.751524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb096a-be08-4f20-bd83-cfdcdb3ae1fa service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:31.751885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb096a-be08-4f20-bd83-cfdcdb3ae1fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:31.753540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a1cb096a-be08-4f20-bd83-cfdcdb3ae1fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:31.847280 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb096a-be08-4f20-bd83-cfdcdb3ae1fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:31.848913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb096a-be08-4f20-bd83-cfdcdb3ae1fa service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:31.867645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 712/1416] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:32.773560 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499b51d-ba9f-4461-b597-b694dadccc0a None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:32.774148 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499b51d-ba9f-4461-b597-b694dadccc0a None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:32.774148 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499b51d-ba9f-4461-b597-b694dadccc0a None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:32.775499 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:32.780321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df8eea-9994-493b-9101-b639356ea69a None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:32.780738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df8eea-9994-493b-9101-b639356ea69a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:32.780877 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df8eea-9994-493b-9101-b639356ea69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:32.859242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-36df8eea-9994-493b-9101-b639356ea69a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:32.860462 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-36df8eea-9994-493b-9101-b639356ea69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:32.887007 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:32.956773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:33.112481 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499b51d-ba9f-4461-b597-b694dadccc0a None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:33.113989 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-6499b51d-ba9f-4461-b597-b694dadccc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:33.273972 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499b51d-ba9f-4461-b597-b694dadccc0a None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:33.316388 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:33.389095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b35f32-b571-4647-9a0d-44f3bb058d29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:33.389626 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b35f32-b571-4647-9a0d-44f3bb058d29 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:33.389901 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b35f32-b571-4647-9a0d-44f3bb058d29 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:33.390174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b35f32-b571-4647-9a0d-44f3bb058d29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:33.391802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-03b35f32-b571-4647-9a0d-44f3bb058d29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:33.483082 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b35f32-b571-4647-9a0d-44f3bb058d29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:33.484850 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b35f32-b571-4647-9a0d-44f3bb058d29 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:33.503604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 706/1419] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:34.388183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b90b7f-b395-4401-8a13-8913083b87c0 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:34.388869 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b90b7f-b395-4401-8a13-8913083b87c0 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:34.388869 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b90b7f-b395-4401-8a13-8913083b87c0 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:34.390103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:34.395906 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb5b62-a413-4610-8fa9-bcdc0fd15951 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:34.398576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb5b62-a413-4610-8fa9-bcdc0fd15951 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:34.398576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb5b62-a413-4610-8fa9-bcdc0fd15951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:34.453174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:34.477908 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-d9fb5b62-a413-4610-8fa9-bcdc0fd15951 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:34.479201 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d9fb5b62-a413-4610-8fa9-bcdc0fd15951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:34.578311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:34.732180 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b90b7f-b395-4401-8a13-8913083b87c0 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:34.733689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e4b90b7f-b395-4401-8a13-8913083b87c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:34.889386 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b90b7f-b395-4401-8a13-8913083b87c0 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:34.902500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:35.006070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66cbd6-47ae-4b6b-a74d-967d8942eb8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:35.006686 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66cbd6-47ae-4b6b-a74d-967d8942eb8f service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:35.006996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66cbd6-47ae-4b6b-a74d-967d8942eb8f service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:35.007273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66cbd6-47ae-4b6b-a74d-967d8942eb8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:35.008892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ff66cbd6-47ae-4b6b-a74d-967d8942eb8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:35.099542 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66cbd6-47ae-4b6b-a74d-967d8942eb8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:35.101152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66cbd6-47ae-4b6b-a74d-967d8942eb8f service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:35.119679 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 715/1422] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:35.993942 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a253032-61c6-4203-8c38-11585ab3bb55 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:35.994525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a253032-61c6-4203-8c38-11585ab3bb55 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:35.994525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a253032-61c6-4203-8c38-11585ab3bb55 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:35.995885 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:36.000802 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613757b-976a-4832-b2b4-822c17877427 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:36.001214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613757b-976a-4832-b2b4-822c17877427 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:36.001355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613757b-976a-4832-b2b4-822c17877427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:36.022807 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:36.080699 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a613757b-976a-4832-b2b4-822c17877427 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:36.081817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a613757b-976a-4832-b2b4-822c17877427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:36.184193 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:36.341141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a253032-61c6-4203-8c38-11585ab3bb55 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:36.342687 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-1a253032-61c6-4203-8c38-11585ab3bb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:36.459970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:36.496630 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a253032-61c6-4203-8c38-11585ab3bb55 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:36.627926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00f79e-b12e-4973-9ef5-0a08d211f2a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:36.628554 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00f79e-b12e-4973-9ef5-0a08d211f2a6 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:36.628834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00f79e-b12e-4973-9ef5-0a08d211f2a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:36.629110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00f79e-b12e-4973-9ef5-0a08d211f2a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:36.630714 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8e00f79e-b12e-4973-9ef5-0a08d211f2a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:36.713150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:36.726793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e00f79e-b12e-4973-9ef5-0a08d211f2a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:36.728429 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e00f79e-b12e-4973-9ef5-0a08d211f2a6 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:36.746744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 709/1425] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:37.628276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb6f56-3540-433c-aa3e-32cd7f678eac None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:37.628872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb6f56-3540-433c-aa3e-32cd7f678eac None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:37.628872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb6f56-3540-433c-aa3e-32cd7f678eac None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:37.630179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:37.635371 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05364df2-5eb9-406e-8c88-6ccc835ce21d None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:37.635814 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05364df2-5eb9-406e-8c88-6ccc835ce21d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:37.635939 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05364df2-5eb9-406e-8c88-6ccc835ce21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:37.721328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-05364df2-5eb9-406e-8c88-6ccc835ce21d None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:37.722588 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-05364df2-5eb9-406e-8c88-6ccc835ce21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:37.822956 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:37.977661 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb6f56-3540-433c-aa3e-32cd7f678eac None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:37.979104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-38cb6f56-3540-433c-aa3e-32cd7f678eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:38.063404 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:38.132143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb6f56-3540-433c-aa3e-32cd7f678eac None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:38.249096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094f540-78dd-4fa3-a72d-308a7dbea0db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:38.249633 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094f540-78dd-4fa3-a72d-308a7dbea0db service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:38.249916 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094f540-78dd-4fa3-a72d-308a7dbea0db service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:38.250191 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094f540-78dd-4fa3-a72d-308a7dbea0db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:38.251997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-0094f540-78dd-4fa3-a72d-308a7dbea0db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:38.302167 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:38.343885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094f540-78dd-4fa3-a72d-308a7dbea0db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:38.346194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094f540-78dd-4fa3-a72d-308a7dbea0db service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:38.365799 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 718/1428] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:39.255776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ce9dd-9653-4e42-9d90-8d1afca7a31e None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:39.256354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ce9dd-9653-4e42-9d90-8d1afca7a31e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:39.256354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ce9dd-9653-4e42-9d90-8d1afca7a31e None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:39.257711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:39.262751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fb29b-eb62-42f9-9837-6336114cab65 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:39.263197 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fb29b-eb62-42f9-9837-6336114cab65 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:39.263568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fb29b-eb62-42f9-9837-6336114cab65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:39.345887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-cc9fb29b-eb62-42f9-9837-6336114cab65 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:39.347168 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-cc9fb29b-eb62-42f9-9837-6336114cab65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:39.446384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:39.601108 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ce9dd-9653-4e42-9d90-8d1afca7a31e None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:39.602627 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-9d0ce9dd-9653-4e42-9d90-8d1afca7a31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:39.655532 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:39.762758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ce9dd-9653-4e42-9d90-8d1afca7a31e None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:39.882931 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5afd6b-576d-4fa1-b4f8-5e68d140f9aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:39.883473 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5afd6b-576d-4fa1-b4f8-5e68d140f9aa service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:39.883757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5afd6b-576d-4fa1-b4f8-5e68d140f9aa service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:39.884034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5afd6b-576d-4fa1-b4f8-5e68d140f9aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:39.885756 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-9e5afd6b-576d-4fa1-b4f8-5e68d140f9aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:39.893770 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:39.980845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5afd6b-576d-4fa1-b4f8-5e68d140f9aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:39.982702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5afd6b-576d-4fa1-b4f8-5e68d140f9aa service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:40.001181 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 712/1431] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:40.906127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a36bad-93f0-4158-88f5-ab781fac5415 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:40.906902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a36bad-93f0-4158-88f5-ab781fac5415 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:40.906902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a36bad-93f0-4158-88f5-ab781fac5415 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:40.908650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:18:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:40.917504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edecac-6008-4070-b229-2c8d4cafcfcf None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:40.918756 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edecac-6008-4070-b229-2c8d4cafcfcf None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:40.918756 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edecac-6008-4070-b229-2c8d4cafcfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:41.118516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-84edecac-6008-4070-b229-2c8d4cafcfcf None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:41.119793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-84edecac-6008-4070-b229-2c8d4cafcfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:41.195185 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:41.224941 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:41.376902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a36bad-93f0-4158-88f5-ab781fac5415 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:41.378642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c7a36bad-93f0-4158-88f5-ab781fac5415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:41.397711 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:41.542402 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a36bad-93f0-4158-88f5-ab781fac5415 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:41.634687 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:41.658526 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf8faf-9ea7-4311-9de4-bce23eeb659e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:41.659172 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf8faf-9ea7-4311-9de4-bce23eeb659e service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:41.659455 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf8faf-9ea7-4311-9de4-bce23eeb659e service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:41.659729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf8faf-9ea7-4311-9de4-bce23eeb659e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:41.661368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-20cf8faf-9ea7-4311-9de4-bce23eeb659e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:41.755807 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf8faf-9ea7-4311-9de4-bce23eeb659e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:41.757422 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf8faf-9ea7-4311-9de4-bce23eeb659e service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:41.774969 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 721/1434] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:42.663369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e99252-6ffe-4eaf-941a-37632a12999d None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:42.664107 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e99252-6ffe-4eaf-941a-37632a12999d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:42.664107 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e99252-6ffe-4eaf-941a-37632a12999d None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:42.665299 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:42.670493 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dbba4-84c7-49cd-95ca-3a710fb6cdf2 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:42.670975 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dbba4-84c7-49cd-95ca-3a710fb6cdf2 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:42.671166 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dbba4-84c7-49cd-95ca-3a710fb6cdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:42.753964 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-648dbba4-84c7-49cd-95ca-3a710fb6cdf2 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:42.755385 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-648dbba4-84c7-49cd-95ca-3a710fb6cdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:42.797991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:42.855951 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:43.013724 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e99252-6ffe-4eaf-941a-37632a12999d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:43.015410 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f2e99252-6ffe-4eaf-941a-37632a12999d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:43.165189 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e99252-6ffe-4eaf-941a-37632a12999d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:43.222848 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:43.282262 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b3e34-2d7f-49dd-8802-3620006e8457 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:43.282898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b3e34-2d7f-49dd-8802-3620006e8457 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:43.283188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b3e34-2d7f-49dd-8802-3620006e8457 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:43.283461 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b3e34-2d7f-49dd-8802-3620006e8457 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:43.285067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-385b3e34-2d7f-49dd-8802-3620006e8457 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:43.377210 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b3e34-2d7f-49dd-8802-3620006e8457 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:43.378831 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b3e34-2d7f-49dd-8802-3620006e8457 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:43.397789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 715/1437] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:44.320888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84298f-ea2d-487f-b8dd-dd9cd406cad7 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:44.321450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84298f-ea2d-487f-b8dd-dd9cd406cad7 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:44.321450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84298f-ea2d-487f-b8dd-dd9cd406cad7 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:44.322791 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:44.327794 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff6616-f462-4c6a-9f6c-95360c75c9ee None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:44.328239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff6616-f462-4c6a-9f6c-95360c75c9ee None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:44.328345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff6616-f462-4c6a-9f6c-95360c75c9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:44.405902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-63ff6616-f462-4c6a-9f6c-95360c75c9ee None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:44.407352 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-63ff6616-f462-4c6a-9f6c-95360c75c9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:44.409746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:44.509803 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:44.665598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84298f-ea2d-487f-b8dd-dd9cd406cad7 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:44.667168 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8f84298f-ea2d-487f-b8dd-dd9cd406cad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:44.815763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84298f-ea2d-487f-b8dd-dd9cd406cad7 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:44.843023 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:44.928740 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ac3fe-107e-4da2-945f-66bb4679d8ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:44.929240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ac3fe-107e-4da2-945f-66bb4679d8ab service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:44.929515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ac3fe-107e-4da2-945f-66bb4679d8ab service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:44.929781 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ac3fe-107e-4da2-945f-66bb4679d8ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:44.931375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f45ac3fe-107e-4da2-945f-66bb4679d8ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:45.021707 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ac3fe-107e-4da2-945f-66bb4679d8ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:45.023341 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ac3fe-107e-4da2-945f-66bb4679d8ab service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:45.041141 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 724/1440] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:46.913886 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799d49e-2f28-4ea7-ba90-05d94330151c None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:46.914453 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799d49e-2f28-4ea7-ba90-05d94330151c None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:46.914453 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799d49e-2f28-4ea7-ba90-05d94330151c None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:46.915842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:46.921100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039a2e2-de7b-4dce-bd78-fc51401849fa None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:46.921557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039a2e2-de7b-4dce-bd78-fc51401849fa None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:46.921655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039a2e2-de7b-4dce-bd78-fc51401849fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:46.962077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:46.988148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a039a2e2-de7b-4dce-bd78-fc51401849fa None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:46.989384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a039a2e2-de7b-4dce-bd78-fc51401849fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:47.091059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:47.388568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799d49e-2f28-4ea7-ba90-05d94330151c None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:47.390119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e799d49e-2f28-4ea7-ba90-05d94330151c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:47.498532 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799d49e-2f28-4ea7-ba90-05d94330151c None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:47.531947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:47.573373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6618b-9b80-4c06-a0cc-60d34ae172bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:47.573978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6618b-9b80-4c06-a0cc-60d34ae172bb service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:47.574257 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6618b-9b80-4c06-a0cc-60d34ae172bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:47.574524 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6618b-9b80-4c06-a0cc-60d34ae172bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:47.576130 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-eba6618b-9b80-4c06-a0cc-60d34ae172bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:47.689776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6618b-9b80-4c06-a0cc-60d34ae172bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:47.691271 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6618b-9b80-4c06-a0cc-60d34ae172bb service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:47.710270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 718/1443] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:49.805568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4487719-ec6b-4b94-9967-5e2e94172223 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:49.806153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4487719-ec6b-4b94-9967-5e2e94172223 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:49.806153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4487719-ec6b-4b94-9967-5e2e94172223 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:49.807505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:49.812961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c53e1-6747-4dca-ba0c-8e865dc05d74 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:49.813438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c53e1-6747-4dca-ba0c-8e865dc05d74 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:49.813553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c53e1-6747-4dca-ba0c-8e865dc05d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:49.883742 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-303c53e1-6747-4dca-ba0c-8e865dc05d74 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:49.884964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-303c53e1-6747-4dca-ba0c-8e865dc05d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:49.898586 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:49.986680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:50.274400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4487719-ec6b-4b94-9967-5e2e94172223 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:50.275887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c4487719-ec6b-4b94-9967-5e2e94172223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:50.381176 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4487719-ec6b-4b94-9967-5e2e94172223 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:50.394142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:50.458194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67955c-6f16-4c80-ae7e-e5461ae9e810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:50.458809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67955c-6f16-4c80-ae7e-e5461ae9e810 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:50.459250 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67955c-6f16-4c80-ae7e-e5461ae9e810 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:50.459390 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67955c-6f16-4c80-ae7e-e5461ae9e810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:50.460996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3c67955c-6f16-4c80-ae7e-e5461ae9e810 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:50.570958 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67955c-6f16-4c80-ae7e-e5461ae9e810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:50.572410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67955c-6f16-4c80-ae7e-e5461ae9e810 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:50.590536 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 727/1446] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:52.934166 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cd21d-b340-47fa-8367-37ae0acb4475 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:52.934750 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cd21d-b340-47fa-8367-37ae0acb4475 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:52.934922 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cd21d-b340-47fa-8367-37ae0acb4475 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:52.936322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:52.942012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecd75f-6cb7-4cf0-961d-6454f41bf8c5 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:52.942412 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecd75f-6cb7-4cf0-961d-6454f41bf8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:52.942569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecd75f-6cb7-4cf0-961d-6454f41bf8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:52.992529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:53.010465 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-93ecd75f-6cb7-4cf0-961d-6454f41bf8c5 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:53.011711 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-93ecd75f-6cb7-4cf0-961d-6454f41bf8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:53.115493 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:53.412227 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044cd21d-b340-47fa-8367-37ae0acb4475 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:53.413763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-044cd21d-b340-47fa-8367-37ae0acb4475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:53.521518 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044cd21d-b340-47fa-8367-37ae0acb4475 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:53.565624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:53.598387 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27eabf1-d9f5-4581-abf7-f8ae8e978a96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:53.599017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27eabf1-d9f5-4581-abf7-f8ae8e978a96 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:53.599294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27eabf1-d9f5-4581-abf7-f8ae8e978a96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:53.599559 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27eabf1-d9f5-4581-abf7-f8ae8e978a96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:53.601135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f27eabf1-d9f5-4581-abf7-f8ae8e978a96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:53.718321 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27eabf1-d9f5-4581-abf7-f8ae8e978a96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:53.719769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27eabf1-d9f5-4581-abf7-f8ae8e978a96 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:53.739522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 721/1449] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:18:55.829887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3306e6-55f1-4619-87a9-5a799a768e5c None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:55.830503 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3306e6-55f1-4619-87a9-5a799a768e5c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:55.830759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3306e6-55f1-4619-87a9-5a799a768e5c None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:55.832177 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:18:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:55.837498 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b0345-6f61-446c-ba4d-4cfdc50f0f4c None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:55.837949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b0345-6f61-446c-ba4d-4cfdc50f0f4c None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:55.838051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b0345-6f61-446c-ba4d-4cfdc50f0f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:55.905512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-d82b0345-6f61-446c-ba4d-4cfdc50f0f4c None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:55.906708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d82b0345-6f61-446c-ba4d-4cfdc50f0f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:55.929490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:56.008346 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:56.299041 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3306e6-55f1-4619-87a9-5a799a768e5c None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:56.300559 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6d3306e6-55f1-4619-87a9-5a799a768e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:56.406973 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3306e6-55f1-4619-87a9-5a799a768e5c None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:56.427532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:56.483680 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f4667-ef57-4e0b-afbb-c4d5a3515d88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:56.484292 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4667-ef57-4e0b-afbb-c4d5a3515d88 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:56.484572 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4667-ef57-4e0b-afbb-c4d5a3515d88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:56.484840 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4667-ef57-4e0b-afbb-c4d5a3515d88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:56.486985 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-031f4667-ef57-4e0b-afbb-c4d5a3515d88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:56.601616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f4667-ef57-4e0b-afbb-c4d5a3515d88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:56.603093 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f4667-ef57-4e0b-afbb-c4d5a3515d88 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:56.622152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 730/1452] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:18:58.719313 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbf111-c07e-4eab-b091-6c3bda47aa50 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:58.719975 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbf111-c07e-4eab-b091-6c3bda47aa50 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:58.719975 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbf111-c07e-4eab-b091-6c3bda47aa50 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:58.721353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:58.726511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5015d6-0bc0-4731-bae9-618f2381dcc5 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:58.726961 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5015d6-0bc0-4731-bae9-618f2381dcc5 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:58.727097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5015d6-0bc0-4731-bae9-618f2381dcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:58.793323 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:58.796645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-1f5015d6-0bc0-4731-bae9-618f2381dcc5 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:58.798567 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-1f5015d6-0bc0-4731-bae9-618f2381dcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:58.898039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:59.190300 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbf111-c07e-4eab-b091-6c3bda47aa50 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:59.191934 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-04bbf111-c07e-4eab-b091-6c3bda47aa50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:59.297582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbf111-c07e-4eab-b091-6c3bda47aa50 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:59.347155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:59.372461 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf63fd0-6b47-4824-9f3e-bc5cba2c4086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:59.373078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf63fd0-6b47-4824-9f3e-bc5cba2c4086 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:59.373351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf63fd0-6b47-4824-9f3e-bc5cba2c4086 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:59.373617 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf63fd0-6b47-4824-9f3e-bc5cba2c4086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:59.375215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-0bf63fd0-6b47-4824-9f3e-bc5cba2c4086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:59.485980 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf63fd0-6b47-4824-9f3e-bc5cba2c4086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:59.487457 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf63fd0-6b47-4824-9f3e-bc5cba2c4086 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:59.506399 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 724/1455] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:01.511071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2080069-44a3-4bc2-b9e6-a01ac73b46ba None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:01.511613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2080069-44a3-4bc2-b9e6-a01ac73b46ba None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:01.511613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2080069-44a3-4bc2-b9e6-a01ac73b46ba None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:01.513109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:19:01.520624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3f46f-d03c-4938-a1b2-3b134fbdc1a4 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:01.521405 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3f46f-d03c-4938-a1b2-3b134fbdc1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:01.521405 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3f46f-d03c-4938-a1b2-3b134fbdc1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:01.588202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-33a3f46f-d03c-4938-a1b2-3b134fbdc1a4 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:01.589436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-33a3f46f-d03c-4938-a1b2-3b134fbdc1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:01.621990 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:01.698470 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:01.985138 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2080069-44a3-4bc2-b9e6-a01ac73b46ba None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:01.986731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c2080069-44a3-4bc2-b9e6-a01ac73b46ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:02.092579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2080069-44a3-4bc2-b9e6-a01ac73b46ba None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:02.121936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:02.168540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13a3b1-cf25-46dd-8d9f-ff9e8a4fce9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:02.169169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13a3b1-cf25-46dd-8d9f-ff9e8a4fce9b service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:02.169450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13a3b1-cf25-46dd-8d9f-ff9e8a4fce9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:02.169724 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13a3b1-cf25-46dd-8d9f-ff9e8a4fce9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:02.171386 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-2a13a3b1-cf25-46dd-8d9f-ff9e8a4fce9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:02.283330 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13a3b1-cf25-46dd-8d9f-ff9e8a4fce9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:02.285020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13a3b1-cf25-46dd-8d9f-ff9e8a4fce9b service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:02.303663 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 733/1458] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:04.333455 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfacbf-fd8b-4221-a0a8-b00fa6a9ac0f None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:04.334078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfacbf-fd8b-4221-a0a8-b00fa6a9ac0f None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:04.334078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfacbf-fd8b-4221-a0a8-b00fa6a9ac0f None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:04.335504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:19:04.340924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf19df-08cd-48e7-bd93-bf07335a4528 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:04.341410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf19df-08cd-48e7-bd93-bf07335a4528 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:04.341527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf19df-08cd-48e7-bd93-bf07335a4528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:04.416188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-8daf19df-08cd-48e7-bd93-bf07335a4528 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:04.417434 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8daf19df-08cd-48e7-bd93-bf07335a4528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:04.426639 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:04.519645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:04.807568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfacbf-fd8b-4221-a0a8-b00fa6a9ac0f None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:04.809083 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-48cfacbf-fd8b-4221-a0a8-b00fa6a9ac0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:04.913523 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfacbf-fd8b-4221-a0a8-b00fa6a9ac0f None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:04.968937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:04.988454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c59fb-0f73-48ed-aff2-c81b02352c5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:04.989077 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c59fb-0f73-48ed-aff2-c81b02352c5c service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:04.989422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c59fb-0f73-48ed-aff2-c81b02352c5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:04.989615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c59fb-0f73-48ed-aff2-c81b02352c5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:04.991286 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-db1c59fb-0f73-48ed-aff2-c81b02352c5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:05.106164 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c59fb-0f73-48ed-aff2-c81b02352c5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:05.107633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c59fb-0f73-48ed-aff2-c81b02352c5c service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:05.126119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 727/1461] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:07.077995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfabbcb-b977-4736-9186-07b96997ab25 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:07.078580 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfabbcb-b977-4736-9186-07b96997ab25 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:07.078580 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfabbcb-b977-4736-9186-07b96997ab25 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:07.079925 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:19:07.085814 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef972c-10ad-484c-ad5f-47e1200ba77d None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:07.086278 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef972c-10ad-484c-ad5f-47e1200ba77d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:07.086386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef972c-10ad-484c-ad5f-47e1200ba77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:07.154360 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-53ef972c-10ad-484c-ad5f-47e1200ba77d None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:07.155647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-53ef972c-10ad-484c-ad5f-47e1200ba77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:07.195103 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:07.253697 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:07.538694 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfabbcb-b977-4736-9186-07b96997ab25 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:07.540233 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-cdfabbcb-b977-4736-9186-07b96997ab25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:07.649349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfabbcb-b977-4736-9186-07b96997ab25 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:07.686779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:07.722861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc36bd7-968b-4ed7-b2ca-f55cea00cd32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:07.723493 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc36bd7-968b-4ed7-b2ca-f55cea00cd32 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:07.723773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc36bd7-968b-4ed7-b2ca-f55cea00cd32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:07.724043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc36bd7-968b-4ed7-b2ca-f55cea00cd32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:07.725687 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9bc36bd7-968b-4ed7-b2ca-f55cea00cd32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:07.835814 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc36bd7-968b-4ed7-b2ca-f55cea00cd32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:07.837256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc36bd7-968b-4ed7-b2ca-f55cea00cd32 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:07.856749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 736/1464] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:09.850380 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281eea0-314f-4359-b7ae-1399db2ee6ec None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:09.850972 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281eea0-314f-4359-b7ae-1399db2ee6ec None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:09.850972 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281eea0-314f-4359-b7ae-1399db2ee6ec None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:09.852304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:19:09.857846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565294c-2865-42d4-979e-7cda2eb33604 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:09.858274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565294c-2865-42d4-979e-7cda2eb33604 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:09.858426 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565294c-2865-42d4-979e-7cda2eb33604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:09.926927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-e565294c-2865-42d4-979e-7cda2eb33604 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:09.928171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e565294c-2865-42d4-979e-7cda2eb33604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:09.946285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:10.026341 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:10.310708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c281eea0-314f-4359-b7ae-1399db2ee6ec None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:10.312305 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c281eea0-314f-4359-b7ae-1399db2ee6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:10.418752 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c281eea0-314f-4359-b7ae-1399db2ee6ec None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:10.487896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:10.494675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fec255-d9ff-4389-a6ab-d3f37e54011e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:10.495313 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fec255-d9ff-4389-a6ab-d3f37e54011e service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:10.495593 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fec255-d9ff-4389-a6ab-d3f37e54011e service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:10.495861 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fec255-d9ff-4389-a6ab-d3f37e54011e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:10.497480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-14fec255-d9ff-4389-a6ab-d3f37e54011e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:10.609944 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fec255-d9ff-4389-a6ab-d3f37e54011e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:10.611534 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fec255-d9ff-4389-a6ab-d3f37e54011e service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:10.630597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 730/1467] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:12.585655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925e179-31cc-4860-8767-2e286b7cc0e7 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:12.586245 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925e179-31cc-4860-8767-2e286b7cc0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:12.586245 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925e179-31cc-4860-8767-2e286b7cc0e7 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:12.587653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:19:12.592929 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98dd8f-0ef1-4c49-928f-31828bc5b48c None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:12.593427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98dd8f-0ef1-4c49-928f-31828bc5b48c None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:12.593537 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98dd8f-0ef1-4c49-928f-31828bc5b48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:12.662488 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-2f98dd8f-0ef1-4c49-928f-31828bc5b48c None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:12.663754 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-2f98dd8f-0ef1-4c49-928f-31828bc5b48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:12.714686 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:12.778829 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:19:13.099409 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925e179-31cc-4860-8767-2e286b7cc0e7 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:13.101368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6925e179-31cc-4860-8767-2e286b7cc0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:13.208579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925e179-31cc-4860-8767-2e286b7cc0e7 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:13.256803 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:13.288951 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59448a0d-8b30-4eb6-b218-54af27089e53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:13.289569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59448a0d-8b30-4eb6-b218-54af27089e53 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:13.289846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59448a0d-8b30-4eb6-b218-54af27089e53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:13.290116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59448a0d-8b30-4eb6-b218-54af27089e53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:13.291747 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-59448a0d-8b30-4eb6-b218-54af27089e53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:13.403601 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59448a0d-8b30-4eb6-b218-54af27089e53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:13.404961 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59448a0d-8b30-4eb6-b218-54af27089e53 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:13.423838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 739/1470] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:22.776767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=22 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:23.422336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=66 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:33.299102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42283a06-e4be-4324-b927-c2ac2e27da9a None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:33.299659 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42283a06-e4be-4324-b927-c2ac2e27da9a None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:33.299659 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42283a06-e4be-4324-b927-c2ac2e27da9a None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:33.300984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 732/1471] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:19:33.306068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ded0e6-24d3-4b7c-8a15-754b8332200e None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:33.306506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ded0e6-24d3-4b7c-8a15-754b8332200e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:33.306623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ded0e6-24d3-4b7c-8a15-754b8332200e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:33.372133 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d2ded0e6-24d3-4b7c-8a15-754b8332200e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:33.373325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d2ded0e6-24d3-4b7c-8a15-754b8332200e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:33.473626 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 740/1472] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:33.767668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8db626-6f10-4a7b-9dce-594ea9bbebbb None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:33.768141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8db626-6f10-4a7b-9dce-594ea9bbebbb None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:33.768423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8db626-6f10-4a7b-9dce-594ea9bbebbb None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:33.769852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 733/1473] 10.222.0.59 () {54 vars in 967 bytes} [Wed Aug 31 19:19:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:19:33.774768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cfd97-048d-4491-9a0f-7b812912ea73 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:33.775205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cfd97-048d-4491-9a0f-7b812912ea73 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:33.775494 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cfd97-048d-4491-9a0f-7b812912ea73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:33.852479 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d72cfd97-048d-4491-9a0f-7b812912ea73 None None] MFA Rules not processed for user `f281d778f1404984b4d778a1c26aef1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:33.853716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d72cfd97-048d-4491-9a0f-7b812912ea73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:33.874624 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:33.929318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 741/1474] 10.222.0.59 () {58 vars in 1064 bytes} [Wed Aug 31 19:19:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:33.936926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8db626-6f10-4a7b-9dce-594ea9bbebbb None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:33.938467 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ba8db626-6f10-4a7b-9dce-594ea9bbebbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:34.044762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8db626-6f10-4a7b-9dce-594ea9bbebbb None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:34.118773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff990cd-ce24-4392-8ef1-e3d732f48cc0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f281d778f1404984b4d778a1c26aef1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:34.119423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff990cd-ce24-4392-8ef1-e3d732f48cc0 service glance] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:34.119717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff990cd-ce24-4392-8ef1-e3d732f48cc0 service glance] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:34.119987 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff990cd-ce24-4392-8ef1-e3d732f48cc0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:34.121580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-dff990cd-ce24-4392-8ef1-e3d732f48cc0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:34.175939 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:34.231642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff990cd-ce24-4392-8ef1-e3d732f48cc0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:34.233088 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff990cd-ce24-4392-8ef1-e3d732f48cc0 service glance] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:34.252451 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 734/1475] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:36.273220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2efd2-aa5d-4586-b3cc-3cf0a6ef0ab4 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:36.273824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2efd2-aa5d-4586-b3cc-3cf0a6ef0ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:36.273824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2efd2-aa5d-4586-b3cc-3cf0a6ef0ab4 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:36.275251 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 742/1476] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:19:36.280591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e55015-4299-4664-b97f-0608910ba80b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:36.281007 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e55015-4299-4664-b97f-0608910ba80b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:36.281139 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e55015-4299-4664-b97f-0608910ba80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:36.347724 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-48e55015-4299-4664-b97f-0608910ba80b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:36.348937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-48e55015-4299-4664-b97f-0608910ba80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:36.446927 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 735/1477] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:19:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:36.728179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2efd2-aa5d-4586-b3cc-3cf0a6ef0ab4 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:36.729709 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-70b2efd2-aa5d-4586-b3cc-3cf0a6ef0ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:36.858232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2efd2-aa5d-4586-b3cc-3cf0a6ef0ab4 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:36.906696 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:36.952655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f3cf3-214d-4351-b971-777ea9d779d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:36.953264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f3cf3-214d-4351-b971-777ea9d779d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:36.953539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f3cf3-214d-4351-b971-777ea9d779d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:36.953805 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f3cf3-214d-4351-b971-777ea9d779d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:36.954557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f3cf3-214d-4351-b971-777ea9d779d3 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:36.956904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: WARNING py.warnings [None req-259f3cf3-214d-4351-b971-777ea9d779d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:19:36.956904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: warnings.warn(msg) Aug 31 19:19:36.956904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: Aug 31 19:19:36.958229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f3cf3-214d-4351-b971-777ea9d779d3 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:36.988431 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 743/1478] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 31 19:19:36 2022] GET /identity/v3/projects => generated 2553 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:38.934122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc0ec4-393b-4a0d-9dce-a6fa54a0a811 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:38.934717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc0ec4-393b-4a0d-9dce-a6fa54a0a811 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:38.934717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc0ec4-393b-4a0d-9dce-a6fa54a0a811 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:38.936032 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 736/1479] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:19:38.942146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8853030-377d-41ec-a11f-dbaf02960064 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:38.942593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8853030-377d-41ec-a11f-dbaf02960064 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:38.942706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8853030-377d-41ec-a11f-dbaf02960064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:39.012423 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a8853030-377d-41ec-a11f-dbaf02960064 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:39.013638 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a8853030-377d-41ec-a11f-dbaf02960064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:39.109618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 744/1480] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:39.406968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc0ec4-393b-4a0d-9dce-a6fa54a0a811 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:39.408576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-47bc0ec4-393b-4a0d-9dce-a6fa54a0a811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:39.418300 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:39.556572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc0ec4-393b-4a0d-9dce-a6fa54a0a811 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:39.637704 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:39.670896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5776cd99-595a-4eb4-85cd-ae8076a3e3ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:39.671534 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776cd99-595a-4eb4-85cd-ae8076a3e3ad service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:39.671812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776cd99-595a-4eb4-85cd-ae8076a3e3ad service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:39.672082 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776cd99-595a-4eb4-85cd-ae8076a3e3ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:39.674288 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-5776cd99-595a-4eb4-85cd-ae8076a3e3ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:39.763330 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5776cd99-595a-4eb4-85cd-ae8076a3e3ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:39.764906 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5776cd99-595a-4eb4-85cd-ae8076a3e3ad service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:39.783536 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 737/1481] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:41.921864 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef13c9-1633-4b7d-90c1-d4be5ba9fb7b None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:41.922456 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef13c9-1633-4b7d-90c1-d4be5ba9fb7b None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:41.922456 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef13c9-1633-4b7d-90c1-d4be5ba9fb7b None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:41.924006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 745/1482] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:19:41.929266 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf442b8-8773-40f2-a726-c48ac444971e None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:41.929692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf442b8-8773-40f2-a726-c48ac444971e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:41.929803 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf442b8-8773-40f2-a726-c48ac444971e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:41.998032 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-4bf442b8-8773-40f2-a726-c48ac444971e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:41.999264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4bf442b8-8773-40f2-a726-c48ac444971e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:42.046001 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:42.101169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 738/1483] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:42.389165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef13c9-1633-4b7d-90c1-d4be5ba9fb7b None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:42.390675 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a9ef13c9-1633-4b7d-90c1-d4be5ba9fb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:42.415064 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:42.495869 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef13c9-1633-4b7d-90c1-d4be5ba9fb7b None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:42.569915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b658cf-1906-41e6-86a3-9a89efdb20e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:42.570525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b658cf-1906-41e6-86a3-9a89efdb20e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:42.570803 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b658cf-1906-41e6-86a3-9a89efdb20e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:42.571103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b658cf-1906-41e6-86a3-9a89efdb20e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:42.572686 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c3b658cf-1906-41e6-86a3-9a89efdb20e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:42.646611 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:42.682805 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b658cf-1906-41e6-86a3-9a89efdb20e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:42.684270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b658cf-1906-41e6-86a3-9a89efdb20e1 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:42.702175 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 746/1484] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:44.651443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132cba9-2edc-4fd9-8f95-bc7970e7eb5a None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:44.651994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132cba9-2edc-4fd9-8f95-bc7970e7eb5a None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:44.651994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132cba9-2edc-4fd9-8f95-bc7970e7eb5a None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:44.653328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 739/1485] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:19:44.658673 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd688e2-c74f-4492-ab06-450fc3c7dae6 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:44.659154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd688e2-c74f-4492-ab06-450fc3c7dae6 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:44.659263 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd688e2-c74f-4492-ab06-450fc3c7dae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:44.726160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-4dd688e2-c74f-4492-ab06-450fc3c7dae6 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:44.727427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4dd688e2-c74f-4492-ab06-450fc3c7dae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:44.824559 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 747/1486] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:45.121871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132cba9-2edc-4fd9-8f95-bc7970e7eb5a None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:45.123423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-1132cba9-2edc-4fd9-8f95-bc7970e7eb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:45.228549 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132cba9-2edc-4fd9-8f95-bc7970e7eb5a None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:45.304556 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:45.306421 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4aeae-b5a5-489c-bb46-d0f972c9f9bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:45.307064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4aeae-b5a5-489c-bb46-d0f972c9f9bb service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:45.307340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4aeae-b5a5-489c-bb46-d0f972c9f9bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:45.307606 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4aeae-b5a5-489c-bb46-d0f972c9f9bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:45.309184 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ebb4aeae-b5a5-489c-bb46-d0f972c9f9bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:45.422338 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4aeae-b5a5-489c-bb46-d0f972c9f9bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:45.423806 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4aeae-b5a5-489c-bb46-d0f972c9f9bb service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:45.442436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 740/1487] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:47.566691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef696889-deb7-42f9-b293-41d0cafb75a9 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:47.567288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef696889-deb7-42f9-b293-41d0cafb75a9 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:47.567288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef696889-deb7-42f9-b293-41d0cafb75a9 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:47.568542 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 748/1488] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:19:47.573826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83903176-563d-4832-93c2-f9bdb499a467 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:47.574367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83903176-563d-4832-93c2-f9bdb499a467 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:47.574367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83903176-563d-4832-93c2-f9bdb499a467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:47.640481 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-83903176-563d-4832-93c2-f9bdb499a467 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:47.641748 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-83903176-563d-4832-93c2-f9bdb499a467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:47.695228 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:47.739436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 741/1489] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:48.029318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef696889-deb7-42f9-b293-41d0cafb75a9 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:48.030896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ef696889-deb7-42f9-b293-41d0cafb75a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:48.068082 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:48.138145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef696889-deb7-42f9-b293-41d0cafb75a9 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:48.211587 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4b1ec-3ffc-470b-bfc3-4bd6d7274571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:48.212212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4b1ec-3ffc-470b-bfc3-4bd6d7274571 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:48.212497 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4b1ec-3ffc-470b-bfc3-4bd6d7274571 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:48.212774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4b1ec-3ffc-470b-bfc3-4bd6d7274571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:48.214368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-91d4b1ec-3ffc-470b-bfc3-4bd6d7274571 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:48.295809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:48.324485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4b1ec-3ffc-470b-bfc3-4bd6d7274571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:48.325941 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4b1ec-3ffc-470b-bfc3-4bd6d7274571 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:48.346281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 749/1490] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:50.726280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7b9a6-bbf9-4928-a374-b062b87d6033 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:50.726897 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7b9a6-bbf9-4928-a374-b062b87d6033 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:50.726897 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7b9a6-bbf9-4928-a374-b062b87d6033 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:50.728212 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 742/1491] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 31 19:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:19:50.733668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba82bf-f763-4e9e-912a-dc7951325df9 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:50.734063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba82bf-f763-4e9e-912a-dc7951325df9 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:50.734229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba82bf-f763-4e9e-912a-dc7951325df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:50.801645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-8cba82bf-f763-4e9e-912a-dc7951325df9 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:50.802903 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8cba82bf-f763-4e9e-912a-dc7951325df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:50.901674 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 750/1492] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 31 19:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:51.190551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7b9a6-bbf9-4928-a374-b062b87d6033 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:51.192128 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e6f7b9a6-bbf9-4928-a374-b062b87d6033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:51.299349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7b9a6-bbf9-4928-a374-b062b87d6033 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:51.371295 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ee41b-765a-4a96-87f5-68ebd2337dfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:51.371907 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ee41b-765a-4a96-87f5-68ebd2337dfe service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:51.372184 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ee41b-765a-4a96-87f5-68ebd2337dfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:51.372453 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ee41b-765a-4a96-87f5-68ebd2337dfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:51.374030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e33ee41b-765a-4a96-87f5-68ebd2337dfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:51.385348 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:51.488675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ee41b-765a-4a96-87f5-68ebd2337dfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:51.490095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ee41b-765a-4a96-87f5-68ebd2337dfe service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:51.507798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 743/1493] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:20:00.899865 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=82 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:01.505472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=50 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:16.160871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27633d47-69bf-4d50-97b7-9bce44a9d6bb None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:16.161504 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27633d47-69bf-4d50-97b7-9bce44a9d6bb None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:16.161504 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27633d47-69bf-4d50-97b7-9bce44a9d6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:16.356210 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-27633d47-69bf-4d50-97b7-9bce44a9d6bb None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:16.357425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-27633d47-69bf-4d50-97b7-9bce44a9d6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:16.453573 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 751/1494] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:16.461525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742746d7-6be9-43f0-8916-c890c6f23b97 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:16.462034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742746d7-6be9-43f0-8916-c890c6f23b97 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:16.462141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742746d7-6be9-43f0-8916-c890c6f23b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:16.538018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-742746d7-6be9-43f0-8916-c890c6f23b97 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:16.539273 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-742746d7-6be9-43f0-8916-c890c6f23b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:16.636333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 744/1495] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:16.641700 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27633d47-69bf-4d50-97b7-9bce44a9d6bb None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:16.642957 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-27633d47-69bf-4d50-97b7-9bce44a9d6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:16.654128 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27633d47-69bf-4d50-97b7-9bce44a9d6bb None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:16.714543 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:16.747641 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417d04c-d5a6-428a-ba23-6685584b6056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:16.748486 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417d04c-d5a6-428a-ba23-6685584b6056 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:16.748958 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417d04c-d5a6-428a-ba23-6685584b6056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:16.750253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417d04c-d5a6-428a-ba23-6685584b6056 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:16.751047 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417d04c-d5a6-428a-ba23-6685584b6056 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:21:16.751359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417d04c-d5a6-428a-ba23-6685584b6056 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:16.752389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: WARNING py.warnings [None req-3417d04c-d5a6-428a-ba23-6685584b6056 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:21:16.752389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: warnings.warn(msg) Aug 31 19:21:16.752389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: Aug 31 19:21:16.753278 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417d04c-d5a6-428a-ba23-6685584b6056 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:16.764521 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 752/1496] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:21:16.773283 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742746d7-6be9-43f0-8916-c890c6f23b97 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:16.782709 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742746d7-6be9-43f0-8916-c890c6f23b97 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:16.844056 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:16.879180 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077a9ce-f1eb-4c26-95b2-61d227b33649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:16.879791 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077a9ce-f1eb-4c26-95b2-61d227b33649 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:16.880072 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077a9ce-f1eb-4c26-95b2-61d227b33649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:16.880344 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077a9ce-f1eb-4c26-95b2-61d227b33649 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:16.880985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077a9ce-f1eb-4c26-95b2-61d227b33649 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:16.882719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: WARNING py.warnings [None req-8077a9ce-f1eb-4c26-95b2-61d227b33649 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:21:16.882719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: warnings.warn(msg) Aug 31 19:21:16.882719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: Aug 31 19:21:16.883974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077a9ce-f1eb-4c26-95b2-61d227b33649 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:17.024166 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 745/1497] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:21:16 2022] POST /identity/v3/projects => generated 379 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:21:17.028489 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417d04c-d5a6-428a-ba23-6685584b6056 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:17.030111 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3417d04c-d5a6-428a-ba23-6685584b6056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:17.064549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417d04c-d5a6-428a-ba23-6685584b6056 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:17.160858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fddc577-b132-48ba-885b-62b8792f8a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:17.161469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddc577-b132-48ba-885b-62b8792f8a1d admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:17.162031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddc577-b132-48ba-885b-62b8792f8a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:17.162140 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddc577-b132-48ba-885b-62b8792f8a1d admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:17.162784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fddc577-b132-48ba-885b-62b8792f8a1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:17.164029 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: WARNING py.warnings [None req-5fddc577-b132-48ba-885b-62b8792f8a1d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:21:17.164029 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: warnings.warn(msg) Aug 31 19:21:17.164029 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: Aug 31 19:21:17.165233 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fddc577-b132-48ba-885b-62b8792f8a1d admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:17.275319 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 753/1498] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:21:17 2022] POST /identity/v3/users => generated 290 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:21:17.279617 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077a9ce-f1eb-4c26-95b2-61d227b33649 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:17.281926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8077a9ce-f1eb-4c26-95b2-61d227b33649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:17.346182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077a9ce-f1eb-4c26-95b2-61d227b33649 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:17.432621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:17.442826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d192c83-0b89-48bd-a199-feb23680c269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:17.443455 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d192c83-0b89-48bd-a199-feb23680c269 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:17.443729 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d192c83-0b89-48bd-a199-feb23680c269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:17.443998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d192c83-0b89-48bd-a199-feb23680c269 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:17.444680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d192c83-0b89-48bd-a199-feb23680c269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:17.445686 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: WARNING py.warnings [None req-1d192c83-0b89-48bd-a199-feb23680c269 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:21:17.445686 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: warnings.warn(msg) Aug 31 19:21:17.445686 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: Aug 31 19:21:17.446563 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d192c83-0b89-48bd-a199-feb23680c269 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:17.460203 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 746/1499] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:21:17.464488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fddc577-b132-48ba-885b-62b8792f8a1d admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:17.465964 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-5fddc577-b132-48ba-885b-62b8792f8a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:17.479939 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fddc577-b132-48ba-885b-62b8792f8a1d admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:17.504601 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:17.574037 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbb7a1-08c6-4f44-894a-81af39c28231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:17.574653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbb7a1-08c6-4f44-894a-81af39c28231 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:17.574968 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbb7a1-08c6-4f44-894a-81af39c28231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:17.575238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbb7a1-08c6-4f44-894a-81af39c28231 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:17.575909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccbb7a1-08c6-4f44-894a-81af39c28231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:17.577914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: WARNING py.warnings [None req-9ccbb7a1-08c6-4f44-894a-81af39c28231 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:21:17.577914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: warnings.warn(msg) Aug 31 19:21:17.577914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: Aug 31 19:21:17.579027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccbb7a1-08c6-4f44-894a-81af39c28231 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:17.591605 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 754/1500] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:21:17.596662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d192c83-0b89-48bd-a199-feb23680c269 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:17.598213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-1d192c83-0b89-48bd-a199-feb23680c269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:17.631943 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d192c83-0b89-48bd-a199-feb23680c269 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:17.727633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769f3a1-e483-4a41-a95c-87d14954c68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:17.728343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769f3a1-e483-4a41-a95c-87d14954c68e admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:17.728629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769f3a1-e483-4a41-a95c-87d14954c68e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:17.728899 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769f3a1-e483-4a41-a95c-87d14954c68e admin admin] PATH_INFO: `/v3/projects/b9e2f1a1ea82429998b179e9dbcc6ffe/users/052bebcea1484c85a56e8edf9211ead8/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:17.751592 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769f3a1-e483-4a41-a95c-87d14954c68e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e2f1a1ea82429998b179e9dbcc6ffe, user_id=052bebcea1484c85a56e8edf9211ead8, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:17.752621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: WARNING py.warnings [None req-2769f3a1-e483-4a41-a95c-87d14954c68e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:21:17.752621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: warnings.warn(msg) Aug 31 19:21:17.752621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: Aug 31 19:21:17.754602 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769f3a1-e483-4a41-a95c-87d14954c68e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:17.899205 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 747/1501] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:21:17 2022] PUT /identity/v3/projects/b9e2f1a1ea82429998b179e9dbcc6ffe/users/052bebcea1484c85a56e8edf9211ead8/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:17.912002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e14d2c-fe7f-45ff-9ff5-6f97466aa413 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:17.917071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e14d2c-fe7f-45ff-9ff5-6f97466aa413 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:17.917071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e14d2c-fe7f-45ff-9ff5-6f97466aa413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:17.971882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-e3e14d2c-fe7f-45ff-9ff5-6f97466aa413 None None] MFA Rules not processed for user `052bebcea1484c85a56e8edf9211ead8`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:17.973094 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e3e14d2c-fe7f-45ff-9ff5-6f97466aa413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:18.051989 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 755/1502] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:21:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:18.067110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6d0ff-f358-4819-bb09-e763c1b9c603 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:18.067397 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6d0ff-f358-4819-bb09-e763c1b9c603 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:18.067720 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6d0ff-f358-4819-bb09-e763c1b9c603 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:18.069105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 748/1503] 10.222.0.59 () {54 vars in 964 bytes} [Wed Aug 31 19:21:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:21:18.074175 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663361c1-0f73-40ea-9a1f-d3c836784976 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:18.074490 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663361c1-0f73-40ea-9a1f-d3c836784976 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:18.074770 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663361c1-0f73-40ea-9a1f-d3c836784976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:18.106532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:18.151773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-663361c1-0f73-40ea-9a1f-d3c836784976 None None] MFA Rules not processed for user `108a698aa3e6477b9ae896d68f22bd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:18.152909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-663361c1-0f73-40ea-9a1f-d3c836784976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:18.229893 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 756/1504] 10.222.0.59 () {58 vars in 1061 bytes} [Wed Aug 31 19:21:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:18.240134 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6d0ff-f358-4819-bb09-e763c1b9c603 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:18.241693 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-24b6d0ff-f358-4819-bb09-e763c1b9c603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:18.293342 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:18.348734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6d0ff-f358-4819-bb09-e763c1b9c603 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:18.425464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f64fd-03e8-4eb8-97ba-d47bb944a8a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:18.426024 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f64fd-03e8-4eb8-97ba-d47bb944a8a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:18.426333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f64fd-03e8-4eb8-97ba-d47bb944a8a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:18.426611 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f64fd-03e8-4eb8-97ba-d47bb944a8a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:18.428247 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ee8f64fd-03e8-4eb8-97ba-d47bb944a8a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:18.526586 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f64fd-03e8-4eb8-97ba-d47bb944a8a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:18.528093 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f64fd-03e8-4eb8-97ba-d47bb944a8a4 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:18.531650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:18.545816 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 749/1505] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:21:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:21:18.591869 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bd484-e768-4d25-ad7c-50d6265b0120 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:18.592164 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bd484-e768-4d25-ad7c-50d6265b0120 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:18.592444 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bd484-e768-4d25-ad7c-50d6265b0120 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:18.594275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 757/1506] 10.222.0.59 () {52 vars in 830 bytes} [Wed Aug 31 19:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:21:18.597802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f64fd-03e8-4eb8-97ba-d47bb944a8a4 service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:18.599053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ee8f64fd-03e8-4eb8-97ba-d47bb944a8a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:18.701139 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f64fd-03e8-4eb8-97ba-d47bb944a8a4 service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:18.798770 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9076d-1ddb-4936-a17c-2858a17ae2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:18.799515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9076d-1ddb-4936-a17c-2858a17ae2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:18.799795 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9076d-1ddb-4936-a17c-2858a17ae2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:18.800066 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9076d-1ddb-4936-a17c-2858a17ae2da admin admin] PATH_INFO: `/v3/users/052bebcea1484c85a56e8edf9211ead8` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:18.812803 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9076d-1ddb-4936-a17c-2858a17ae2da admin admin] RBAC: Authorizing `identity:delete_user(user_id=052bebcea1484c85a56e8edf9211ead8)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:18.813818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: WARNING py.warnings [None req-00e9076d-1ddb-4936-a17c-2858a17ae2da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:21:18.813818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: warnings.warn(msg) Aug 31 19:21:18.813818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: Aug 31 19:21:18.815161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9076d-1ddb-4936-a17c-2858a17ae2da admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:18.824798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:19.020731 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-00e9076d-1ddb-4936-a17c-2858a17ae2da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '052bebcea1484c85a56e8edf9211ead8'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.079949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-00e9076d-1ddb-4936-a17c-2858a17ae2da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '052bebcea1484c85a56e8edf9211ead8'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.080478 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-00e9076d-1ddb-4936-a17c-2858a17ae2da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '052bebcea1484c85a56e8edf9211ead8'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.104359 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-00e9076d-1ddb-4936-a17c-2858a17ae2da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '052bebcea1484c85a56e8edf9211ead8'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.105062 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-00e9076d-1ddb-4936-a17c-2858a17ae2da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '052bebcea1484c85a56e8edf9211ead8'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.124458 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 750/1507] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:21:18 2022] DELETE /identity/v3/users/052bebcea1484c85a56e8edf9211ead8 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:19.128853 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ac320-e69d-44f0-9658-f36d719249b7 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:19.129134 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ac320-e69d-44f0-9658-f36d719249b7 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:19.129403 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ac320-e69d-44f0-9658-f36d719249b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:19.187356 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-8a4ac320-e69d-44f0-9658-f36d719249b7 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:19.188581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8a4ac320-e69d-44f0-9658-f36d719249b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:19.262842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:19.275503 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 758/1508] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:19.285946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9076d-1ddb-4936-a17c-2858a17ae2da admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:19.287231 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-00e9076d-1ddb-4936-a17c-2858a17ae2da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:19.389174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9076d-1ddb-4936-a17c-2858a17ae2da admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:19.446252 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:19.465960 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a2f60-1c22-4f40-8158-0b5195ec74d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:19.466573 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a2f60-1c22-4f40-8158-0b5195ec74d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:19.466851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a2f60-1c22-4f40-8158-0b5195ec74d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:19.467152 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a2f60-1c22-4f40-8158-0b5195ec74d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:19.468927 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e08a2f60-1c22-4f40-8158-0b5195ec74d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:19.586270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a2f60-1c22-4f40-8158-0b5195ec74d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:19.587772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a2f60-1c22-4f40-8158-0b5195ec74d1 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:19.607102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 751/1509] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:21:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:21:20.076028 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ac320-e69d-44f0-9658-f36d719249b7 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:20.077315 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8a4ac320-e69d-44f0-9658-f36d719249b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:20.088957 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ac320-e69d-44f0-9658-f36d719249b7 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:20.187636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531545b-62ae-4a4a-a93a-ce48c43918dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:20.188274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531545b-62ae-4a4a-a93a-ce48c43918dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:20.188553 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531545b-62ae-4a4a-a93a-ce48c43918dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:20.188824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531545b-62ae-4a4a-a93a-ce48c43918dd admin admin] PATH_INFO: `/v3/projects/b9e2f1a1ea82429998b179e9dbcc6ffe` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:20.199501 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531545b-62ae-4a4a-a93a-ce48c43918dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9e2f1a1ea82429998b179e9dbcc6ffe)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:20.200525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: WARNING py.warnings [None req-4531545b-62ae-4a4a-a93a-ce48c43918dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:21:20.200525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: warnings.warn(msg) Aug 31 19:21:20.200525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: Aug 31 19:21:20.201712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531545b-62ae-4a4a-a93a-ce48c43918dd admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:20.337480 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-4531545b-62ae-4a4a-a93a-ce48c43918dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9e2f1a1ea82429998b179e9dbcc6ffe', 'request_id': 'req-4531545b-62ae-4a4a-a93a-ce48c43918dd'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:20.344915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-4531545b-62ae-4a4a-a93a-ce48c43918dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9e2f1a1ea82429998b179e9dbcc6ffe', 'request_id': 'req-4531545b-62ae-4a4a-a93a-ce48c43918dd'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:20.445519 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-4531545b-62ae-4a4a-a93a-ce48c43918dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9e2f1a1ea82429998b179e9dbcc6ffe', 'request_id': 'req-4531545b-62ae-4a4a-a93a-ce48c43918dd'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:20.478604 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-4531545b-62ae-4a4a-a93a-ce48c43918dd admin admin] The token cache is being invalidate because project b9e2f1a1ea82429998b179e9dbcc6ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:21:20.478918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-4531545b-62ae-4a4a-a93a-ce48c43918dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:20.479478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-4531545b-62ae-4a4a-a93a-ce48c43918dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:20.481038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 759/1510] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:21:20 2022] DELETE /identity/v3/projects/b9e2f1a1ea82429998b179e9dbcc6ffe => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:29.604127 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=62 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:30.476861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:22:54.874227 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6f991-073e-456e-8d14-4892564319a2 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:54.874896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6f991-073e-456e-8d14-4892564319a2 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:54.874896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6f991-073e-456e-8d14-4892564319a2 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:54.876136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:22:54.881097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26d33f-00ab-4232-9699-f52ff1a6e5ae None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:54.881513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26d33f-00ab-4232-9699-f52ff1a6e5ae None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:54.881944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26d33f-00ab-4232-9699-f52ff1a6e5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:54.968729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-8a26d33f-00ab-4232-9699-f52ff1a6e5ae None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:22:54.969990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8a26d33f-00ab-4232-9699-f52ff1a6e5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:55.076037 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:22:55.239783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6f991-073e-456e-8d14-4892564319a2 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:55.242125 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f1c6f991-073e-456e-8d14-4892564319a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:55.365843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6f991-073e-456e-8d14-4892564319a2 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:55.371876 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711007c8-2868-4f2c-889c-ffbfbafbce64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:55.372492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711007c8-2868-4f2c-889c-ffbfbafbce64 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:55.372775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711007c8-2868-4f2c-889c-ffbfbafbce64 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:55.373041 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711007c8-2868-4f2c-889c-ffbfbafbce64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:55.382061 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711007c8-2868-4f2c-889c-ffbfbafbce64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:55.384651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711007c8-2868-4f2c-889c-ffbfbafbce64 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:55.394096 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 753/1513] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:22:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:22:56.240415 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178146be-41c8-4548-8ad0-cbf61f3049e3 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:56.240990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178146be-41c8-4548-8ad0-cbf61f3049e3 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:56.240990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178146be-41c8-4548-8ad0-cbf61f3049e3 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:56.242447 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:22:56.247882 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fc389-a4f2-4162-997e-5127705df674 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:56.248296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fc389-a4f2-4162-997e-5127705df674 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:56.248460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fc389-a4f2-4162-997e-5127705df674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:56.268251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-b84fc389-a4f2-4162-997e-5127705df674 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:22:56.269495 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b84fc389-a4f2-4162-997e-5127705df674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:56.278156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:22:56.433843 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178146be-41c8-4548-8ad0-cbf61f3049e3 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:56.449345 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178146be-41c8-4548-8ad0-cbf61f3049e3 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:56.455233 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f14521-39f3-47f0-ab40-a68b287bc493 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:56.455837 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f14521-39f3-47f0-ab40-a68b287bc493 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:56.456119 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f14521-39f3-47f0-ab40-a68b287bc493 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:56.456394 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f14521-39f3-47f0-ab40-a68b287bc493 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:56.465103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f14521-39f3-47f0-ab40-a68b287bc493 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:56.466709 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f14521-39f3-47f0-ab40-a68b287bc493 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:56.476192 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 762/1516] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:22:57.172393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad9355-bab9-49df-9fa0-67f3303e9248 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:57.172685 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad9355-bab9-49df-9fa0-67f3303e9248 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:57.172966 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad9355-bab9-49df-9fa0-67f3303e9248 None None] PATH_INFO: `/v3` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:57.174266 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 755/1517] 10.222.0.17 () {54 vars in 925 bytes} [Wed Aug 31 19:22:57 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:22:57.180660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434b4a8-1751-4fcc-8c45-f70bd976f17d None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:57.180935 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434b4a8-1751-4fcc-8c45-f70bd976f17d None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:57.181203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434b4a8-1751-4fcc-8c45-f70bd976f17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:57.241710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d434b4a8-1751-4fcc-8c45-f70bd976f17d None None] MFA Rules not processed for user `eced13cc4030423abe63bb1527f6d35a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:22:57.242997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d434b4a8-1751-4fcc-8c45-f70bd976f17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:57.259760 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:22:57.316168 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 763/1518] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Aug 31 19:22:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:06.263349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=56 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:06.872643 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b0f27-afed-440d-a5cc-1a6ee9143e67 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:06.872856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b0f27-afed-440d-a5cc-1a6ee9143e67 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:06.873101 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b0f27-afed-440d-a5cc-1a6ee9143e67 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:06.874464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 756/1519] 10.222.0.59 () {54 vars in 961 bytes} [Wed Aug 31 19:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:23:06.880398 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fb829-2882-4f59-802f-90f2e5328cbc None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:06.880925 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fb829-2882-4f59-802f-90f2e5328cbc None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:06.880925 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fb829-2882-4f59-802f-90f2e5328cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:06.901537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-6a5fb829-2882-4f59-802f-90f2e5328cbc None None] MFA Rules not processed for user `eced13cc4030423abe63bb1527f6d35a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:06.902831 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6a5fb829-2882-4f59-802f-90f2e5328cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:06.911758 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 764/1520] 10.222.0.59 () {58 vars in 1058 bytes} [Wed Aug 31 19:23:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:06.919350 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b0f27-afed-440d-a5cc-1a6ee9143e67 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:06.933362 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b0f27-afed-440d-a5cc-1a6ee9143e67 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:06.939299 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b81715-2b08-49b0-b178-44b88513d7f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:06.939929 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b81715-2b08-49b0-b178-44b88513d7f7 service placement] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:06.940209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b81715-2b08-49b0-b178-44b88513d7f7 service placement] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:06.940475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b81715-2b08-49b0-b178-44b88513d7f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:06.949275 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b81715-2b08-49b0-b178-44b88513d7f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:06.950886 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b81715-2b08-49b0-b178-44b88513d7f7 service placement] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:06.961536 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 757/1521] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:23:07.757480 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce7a99-dc14-40e8-8f15-263710de469d None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:07.757759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce7a99-dc14-40e8-8f15-263710de469d None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:07.758121 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce7a99-dc14-40e8-8f15-263710de469d None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:07.759604 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:23:07.764908 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30d9f5-3d64-4029-b836-c2c55f0aca9b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:07.765482 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30d9f5-3d64-4029-b836-c2c55f0aca9b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:07.765482 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30d9f5-3d64-4029-b836-c2c55f0aca9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:07.786797 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-4b30d9f5-3d64-4029-b836-c2c55f0aca9b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:07.788126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4b30d9f5-3d64-4029-b836-c2c55f0aca9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:07.797739 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:07.960776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce7a99-dc14-40e8-8f15-263710de469d None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:07.975480 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce7a99-dc14-40e8-8f15-263710de469d None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:07.981331 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b2c8f-e19a-47d8-b6ea-a52e629c92dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:07.981937 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b2c8f-e19a-47d8-b6ea-a52e629c92dc service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:07.982220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b2c8f-e19a-47d8-b6ea-a52e629c92dc service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:07.982489 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b2c8f-e19a-47d8-b6ea-a52e629c92dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:07.991739 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b2c8f-e19a-47d8-b6ea-a52e629c92dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:07.993341 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b2c8f-e19a-47d8-b6ea-a52e629c92dc service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:08.003438 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 766/1524] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:23:08.847993 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a88d3c-da7c-4cb2-b048-089c1f92d051 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:08.848545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a88d3c-da7c-4cb2-b048-089c1f92d051 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:08.848545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a88d3c-da7c-4cb2-b048-089c1f92d051 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:08.849895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:23:08.854968 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a30931-cc1f-41c3-ad93-deda57e7d4e9 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:08.855255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a30931-cc1f-41c3-ad93-deda57e7d4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:08.855527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a30931-cc1f-41c3-ad93-deda57e7d4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:08.876358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-66a30931-cc1f-41c3-ad93-deda57e7d4e9 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:08.878401 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-66a30931-cc1f-41c3-ad93-deda57e7d4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:08.886949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:09.044414 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a88d3c-da7c-4cb2-b048-089c1f92d051 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:09.059727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a88d3c-da7c-4cb2-b048-089c1f92d051 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:09.065401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36fa82-499d-48f5-933f-255cf847b88e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:09.066006 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36fa82-499d-48f5-933f-255cf847b88e service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:09.066285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36fa82-499d-48f5-933f-255cf847b88e service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:09.066561 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36fa82-499d-48f5-933f-255cf847b88e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:09.075401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36fa82-499d-48f5-933f-255cf847b88e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:09.076974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36fa82-499d-48f5-933f-255cf847b88e service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:09.086768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 760/1527] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:12.844750 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe46a4-4057-4519-a754-a3a02455e7a3 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:12.845391 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe46a4-4057-4519-a754-a3a02455e7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:12.845391 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe46a4-4057-4519-a754-a3a02455e7a3 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:12.846683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:23:12.851931 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea8d48-d0e1-4ece-b12d-21c95d87780f None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:12.852360 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea8d48-d0e1-4ece-b12d-21c95d87780f None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:12.852460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea8d48-d0e1-4ece-b12d-21c95d87780f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:12.871984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-74ea8d48-d0e1-4ece-b12d-21c95d87780f None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:12.873167 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-74ea8d48-d0e1-4ece-b12d-21c95d87780f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:12.881505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:13.173551 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe46a4-4057-4519-a754-a3a02455e7a3 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:13.175344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-99fe46a4-4057-4519-a754-a3a02455e7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:13.254990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe46a4-4057-4519-a754-a3a02455e7a3 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:13.258553 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806590f-8370-423e-aae0-500824712993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:13.259187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806590f-8370-423e-aae0-500824712993 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:13.259465 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806590f-8370-423e-aae0-500824712993 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:13.259733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806590f-8370-423e-aae0-500824712993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:13.268634 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806590f-8370-423e-aae0-500824712993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:13.270077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806590f-8370-423e-aae0-500824712993 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:13.280401 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 769/1530] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:15.192665 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1896396-5323-4cae-b17f-24f08cd65ad0 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:15.193247 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1896396-5323-4cae-b17f-24f08cd65ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:15.193247 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1896396-5323-4cae-b17f-24f08cd65ad0 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:15.194572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:23:15.199829 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9df35e-81da-4c6c-85d8-6e81b052c1b8 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:15.200288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9df35e-81da-4c6c-85d8-6e81b052c1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:15.200392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9df35e-81da-4c6c-85d8-6e81b052c1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:15.221817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-dd9df35e-81da-4c6c-85d8-6e81b052c1b8 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:15.223054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-dd9df35e-81da-4c6c-85d8-6e81b052c1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:15.231368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:15.536583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1896396-5323-4cae-b17f-24f08cd65ad0 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:15.548134 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1896396-5323-4cae-b17f-24f08cd65ad0 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:15.551965 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04d5f1-ca42-4d02-be47-c8adc2179340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:15.552580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04d5f1-ca42-4d02-be47-c8adc2179340 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:15.552868 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04d5f1-ca42-4d02-be47-c8adc2179340 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:15.553147 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04d5f1-ca42-4d02-be47-c8adc2179340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:15.564548 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e04d5f1-ca42-4d02-be47-c8adc2179340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:15.565997 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e04d5f1-ca42-4d02-be47-c8adc2179340 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:15.576353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 763/1533] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:19.081679 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1158f8-2548-460a-a0f2-ca8372c8f4f1 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:19.082319 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1158f8-2548-460a-a0f2-ca8372c8f4f1 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:19.082319 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1158f8-2548-460a-a0f2-ca8372c8f4f1 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:19.083583 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:23:19.088730 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53a3b9-6770-4480-b7ae-cfa6a94d52e6 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:19.089154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53a3b9-6770-4480-b7ae-cfa6a94d52e6 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:19.089280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53a3b9-6770-4480-b7ae-cfa6a94d52e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:19.108994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-fe53a3b9-6770-4480-b7ae-cfa6a94d52e6 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:19.110701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-fe53a3b9-6770-4480-b7ae-cfa6a94d52e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:19.118621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:19.668702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1158f8-2548-460a-a0f2-ca8372c8f4f1 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:19.670357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-eb1158f8-2548-460a-a0f2-ca8372c8f4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:19.736140 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:19.749615 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1158f8-2548-460a-a0f2-ca8372c8f4f1 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:19.753860 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52ba0c-581e-42f7-8ad6-145881bdbfde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:19.754465 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52ba0c-581e-42f7-8ad6-145881bdbfde service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:19.754749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52ba0c-581e-42f7-8ad6-145881bdbfde service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:19.755070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52ba0c-581e-42f7-8ad6-145881bdbfde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:19.764273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52ba0c-581e-42f7-8ad6-145881bdbfde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:19.766077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52ba0c-581e-42f7-8ad6-145881bdbfde service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:19.775457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 772/1536] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:21.752936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b9f6e-59ed-4949-a253-fa590f7e4513 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:21.753476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b9f6e-59ed-4949-a253-fa590f7e4513 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:21.753604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b9f6e-59ed-4949-a253-fa590f7e4513 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:21.755163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:23:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:23:21.760377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a356b-0cea-472b-bcdd-fe5d2819dbdb None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:21.760876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a356b-0cea-472b-bcdd-fe5d2819dbdb None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:21.760987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a356b-0cea-472b-bcdd-fe5d2819dbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:21.781247 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-768a356b-0cea-472b-bcdd-fe5d2819dbdb None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:21.784913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-768a356b-0cea-472b-bcdd-fe5d2819dbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:21.790649 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:23:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:22.343460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b9f6e-59ed-4949-a253-fa590f7e4513 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:22.345880 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ea4b9f6e-59ed-4949-a253-fa590f7e4513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:22.359760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b9f6e-59ed-4949-a253-fa590f7e4513 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:22.364435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a1ffa-5b25-4b5c-be92-12c311f23621 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:22.365041 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a1ffa-5b25-4b5c-be92-12c311f23621 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:22.365324 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a1ffa-5b25-4b5c-be92-12c311f23621 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:22.365598 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a1ffa-5b25-4b5c-be92-12c311f23621 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:22.375246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a1ffa-5b25-4b5c-be92-12c311f23621 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:22.376893 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a1ffa-5b25-4b5c-be92-12c311f23621 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:22.387888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 766/1539] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:31.776316 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=11 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:32.382507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=33 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:50.257384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716de2ef-c36b-4552-8952-5d16750edd08 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.259171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716de2ef-c36b-4552-8952-5d16750edd08 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.259171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716de2ef-c36b-4552-8952-5d16750edd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.290696 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-716de2ef-c36b-4552-8952-5d16750edd08 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:50.290696 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-716de2ef-c36b-4552-8952-5d16750edd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:50.308864 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 774/1540] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:50.319192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a5c3d-9f7d-49e8-8870-c0cd4d34162e None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.320295 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a5c3d-9f7d-49e8-8870-c0cd4d34162e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.320798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a5c3d-9f7d-49e8-8870-c0cd4d34162e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.353280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-e45a5c3d-9f7d-49e8-8870-c0cd4d34162e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:50.354565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e45a5c3d-9f7d-49e8-8870-c0cd4d34162e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:50.370735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 767/1541] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:50.376143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716de2ef-c36b-4552-8952-5d16750edd08 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:50.392870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716de2ef-c36b-4552-8952-5d16750edd08 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:50.404740 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181797b-25a4-4fab-8258-19a149c1c6db None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.404990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0c0fc-32c2-4e94-b717-e8b62801f0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:50.405389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0c0fc-32c2-4e94-b717-e8b62801f0ef admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.405676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0c0fc-32c2-4e94-b717-e8b62801f0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.405954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0c0fc-32c2-4e94-b717-e8b62801f0ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.406329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181797b-25a4-4fab-8258-19a149c1c6db None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.406626 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0c0fc-32c2-4e94-b717-e8b62801f0ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:24:50.406970 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0c0fc-32c2-4e94-b717-e8b62801f0ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:50.407517 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181797b-25a4-4fab-8258-19a149c1c6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.408679 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0c0fc-32c2-4e94-b717-e8b62801f0ef admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:50.438192 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 775/1542] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:24:50.438192 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c9051-42bb-4faa-b05a-20b81a4e1fd7 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.438192 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c9051-42bb-4faa-b05a-20b81a4e1fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.438192 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c9051-42bb-4faa-b05a-20b81a4e1fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.441354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-9181797b-25a4-4fab-8258-19a149c1c6db None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:50.442971 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-9181797b-25a4-4fab-8258-19a149c1c6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:50.464046 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 768/1543] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:50.465279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181797b-25a4-4fab-8258-19a149c1c6db None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:50.483585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181797b-25a4-4fab-8258-19a149c1c6db None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:50.488933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcadb84-a553-41de-810e-8bf3eacc04d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:50.489871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcadb84-a553-41de-810e-8bf3eacc04d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.490587 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcadb84-a553-41de-810e-8bf3eacc04d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.491142 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcadb84-a553-41de-810e-8bf3eacc04d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.491942 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-2a6c9051-42bb-4faa-b05a-20b81a4e1fd7 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:50.492294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcadb84-a553-41de-810e-8bf3eacc04d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:50.493168 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-2a6c9051-42bb-4faa-b05a-20b81a4e1fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:50.494787 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcadb84-a553-41de-810e-8bf3eacc04d9 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:50.516754 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 776/1544] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:50.518442 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146c4dc-c2b3-4ca1-87cc-8bafd2dc8427 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.518729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146c4dc-c2b3-4ca1-87cc-8bafd2dc8427 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.519031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146c4dc-c2b3-4ca1-87cc-8bafd2dc8427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.556039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-6146c4dc-c2b3-4ca1-87cc-8bafd2dc8427 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:50.557257 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6146c4dc-c2b3-4ca1-87cc-8bafd2dc8427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:50.571847 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 777/1545] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:50.573129 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13134c0-d4d3-44b8-b302-7f666c119e57 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.573419 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13134c0-d4d3-44b8-b302-7f666c119e57 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.573692 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13134c0-d4d3-44b8-b302-7f666c119e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.608271 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-e13134c0-d4d3-44b8-b302-7f666c119e57 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:50.609502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 769/1546] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:50.609699 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e13134c0-d4d3-44b8-b302-7f666c119e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:50.615628 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc974407-8978-4508-913e-c32adc6a317d None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.618331 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc974407-8978-4508-913e-c32adc6a317d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.618331 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc974407-8978-4508-913e-c32adc6a317d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.649873 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-dc974407-8978-4508-913e-c32adc6a317d None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:50.651421 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-dc974407-8978-4508-913e-c32adc6a317d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:50.707643 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 778/1547] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:50.708756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13134c0-d4d3-44b8-b302-7f666c119e57 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:50.719302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13134c0-d4d3-44b8-b302-7f666c119e57 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:50.724181 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ce103-7ab7-4b94-a87e-d16560a13fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:50.724785 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ce103-7ab7-4b94-a87e-d16560a13fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.725071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ce103-7ab7-4b94-a87e-d16560a13fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.725351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ce103-7ab7-4b94-a87e-d16560a13fc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.726016 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ce103-7ab7-4b94-a87e-d16560a13fc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:24:50.726310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ce103-7ab7-4b94-a87e-d16560a13fc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:50.728219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ce103-7ab7-4b94-a87e-d16560a13fc3 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:50.733305 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 770/1548] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:50.736192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc974407-8978-4508-913e-c32adc6a317d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:50.741631 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 779/1549] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:24:50.743388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabc971-7a28-409b-9adf-a2f21e1aef84 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.743588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabc971-7a28-409b-9adf-a2f21e1aef84 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.743867 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabc971-7a28-409b-9adf-a2f21e1aef84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.745391 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc974407-8978-4508-913e-c32adc6a317d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:50.750367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9fa02-c3a4-4341-98a6-fd78efd5cc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:50.751018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9fa02-c3a4-4341-98a6-fd78efd5cc0f admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.751303 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9fa02-c3a4-4341-98a6-fd78efd5cc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.751578 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9fa02-c3a4-4341-98a6-fd78efd5cc0f admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.752208 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9fa02-c3a4-4341-98a6-fd78efd5cc0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:50.753228 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: WARNING py.warnings [None req-6fb9fa02-c3a4-4341-98a6-fd78efd5cc0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:24:50.753228 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: warnings.warn(msg) Aug 31 19:24:50.753228 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: Aug 31 19:24:50.754426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9fa02-c3a4-4341-98a6-fd78efd5cc0f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:50.764172 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-9dabc971-7a28-409b-9adf-a2f21e1aef84 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:50.765423 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9dabc971-7a28-409b-9adf-a2f21e1aef84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:50.778541 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 780/1550] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:50.778541 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054882d5-da4d-4c59-967d-44bf9f76fca9 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.778541 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054882d5-da4d-4c59-967d-44bf9f76fca9 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.778541 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054882d5-da4d-4c59-967d-44bf9f76fca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.795267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-054882d5-da4d-4c59-967d-44bf9f76fca9 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:50.796590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-054882d5-da4d-4c59-967d-44bf9f76fca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:50.804653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 781/1551] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:50.812959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d66a0a-85d2-42cc-9ab7-dde12e8bc4de None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.812959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d66a0a-85d2-42cc-9ab7-dde12e8bc4de None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.812959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d66a0a-85d2-42cc-9ab7-dde12e8bc4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.826478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-51d66a0a-85d2-42cc-9ab7-dde12e8bc4de None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:50.827704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-51d66a0a-85d2-42cc-9ab7-dde12e8bc4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:50.835908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 782/1552] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:50.841830 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c21de-d2b7-48ee-aa3e-0e02e441638c None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.841830 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c21de-d2b7-48ee-aa3e-0e02e441638c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.841830 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c21de-d2b7-48ee-aa3e-0e02e441638c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.857045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a88c21de-d2b7-48ee-aa3e-0e02e441638c None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:50.858330 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a88c21de-d2b7-48ee-aa3e-0e02e441638c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:50.866378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 783/1553] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:50.869866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 771/1554] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/users => generated 286 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:50.869866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9fa02-c3a4-4341-98a6-fd78efd5cc0f admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:50.887125 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328d980-983c-4e01-bd2f-bc3185be3eeb None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.887125 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328d980-983c-4e01-bd2f-bc3185be3eeb None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.887125 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328d980-983c-4e01-bd2f-bc3185be3eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.887459 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9fa02-c3a4-4341-98a6-fd78efd5cc0f admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:50.887459 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c81751-c2cc-4ea6-b8db-b29760fc36d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:50.887459 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c81751-c2cc-4ea6-b8db-b29760fc36d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.887459 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c81751-c2cc-4ea6-b8db-b29760fc36d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.887886 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c81751-c2cc-4ea6-b8db-b29760fc36d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.888914 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c81751-c2cc-4ea6-b8db-b29760fc36d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:24:50.889216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-f328d980-983c-4e01-bd2f-bc3185be3eeb None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:50.889366 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c81751-c2cc-4ea6-b8db-b29760fc36d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:50.890241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: WARNING py.warnings [None req-67c81751-c2cc-4ea6-b8db-b29760fc36d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:24:50.890241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: warnings.warn(msg) Aug 31 19:24:50.890241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: Aug 31 19:24:50.890709 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f328d980-983c-4e01-bd2f-bc3185be3eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:50.891162 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c81751-c2cc-4ea6-b8db-b29760fc36d2 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:50.901090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 784/1555] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:50.902019 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328d980-983c-4e01-bd2f-bc3185be3eeb None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:50.912135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 772/1556] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:24:50.912135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716542d6-1289-4a89-94db-4ecc6735562f None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.912135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716542d6-1289-4a89-94db-4ecc6735562f None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.912135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716542d6-1289-4a89-94db-4ecc6735562f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.917683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328d980-983c-4e01-bd2f-bc3185be3eeb None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:50.922415 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84169286-8851-40b7-be93-1651b09c876b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:50.923049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84169286-8851-40b7-be93-1651b09c876b admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.923328 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84169286-8851-40b7-be93-1651b09c876b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.923598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84169286-8851-40b7-be93-1651b09c876b admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.924227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84169286-8851-40b7-be93-1651b09c876b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:50.925237 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: WARNING py.warnings [None req-84169286-8851-40b7-be93-1651b09c876b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:24:50.925237 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: warnings.warn(msg) Aug 31 19:24:50.925237 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: Aug 31 19:24:50.926098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-716542d6-1289-4a89-94db-4ecc6735562f None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:50.926419 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84169286-8851-40b7-be93-1651b09c876b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:50.927341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-716542d6-1289-4a89-94db-4ecc6735562f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:50.935881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 773/1557] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:50.937085 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f7514-963d-446e-9d69-12bc0d7e0ee5 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.937376 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f7514-963d-446e-9d69-12bc0d7e0ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.937655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f7514-963d-446e-9d69-12bc0d7e0ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.957602 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-453f7514-963d-446e-9d69-12bc0d7e0ee5 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:50.967916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-453f7514-963d-446e-9d69-12bc0d7e0ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:50.967916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 774/1558] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:50.967916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf65f82-83ea-4bee-b40b-f3926e6b7e6a None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:50.968312 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf65f82-83ea-4bee-b40b-f3926e6b7e6a None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:50.968594 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf65f82-83ea-4bee-b40b-f3926e6b7e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:50.987498 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ebf65f82-83ea-4bee-b40b-f3926e6b7e6a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:50.988678 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ebf65f82-83ea-4bee-b40b-f3926e6b7e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:50.996610 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 775/1559] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:50.997979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf65f82-83ea-4bee-b40b-f3926e6b7e6a None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:51.007230 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf65f82-83ea-4bee-b40b-f3926e6b7e6a None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:51.011755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b3d75-e543-4b55-a6f9-4652cacbb53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:51.012332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b3d75-e543-4b55-a6f9-4652cacbb53b admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:51.012614 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b3d75-e543-4b55-a6f9-4652cacbb53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:51.012985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b3d75-e543-4b55-a6f9-4652cacbb53b admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:51.014218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b3d75-e543-4b55-a6f9-4652cacbb53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:51.015851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b3d75-e543-4b55-a6f9-4652cacbb53b admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:51.024442 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 785/1560] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:50 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:51.026160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34840fad-ca48-4fbc-b63c-be63a2783739 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:51.026449 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34840fad-ca48-4fbc-b63c-be63a2783739 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:51.026721 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34840fad-ca48-4fbc-b63c-be63a2783739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:51.029512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 776/1561] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:51.030295 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b3d75-e543-4b55-a6f9-4652cacbb53b admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:51.040911 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b3d75-e543-4b55-a6f9-4652cacbb53b admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:51.049311 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-34840fad-ca48-4fbc-b63c-be63a2783739 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:51.053779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-34840fad-ca48-4fbc-b63c-be63a2783739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:51.097373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:51.104731 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba644133-bb47-4646-8051-d76a34e08aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:51.105181 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:51.105331 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba644133-bb47-4646-8051-d76a34e08aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:51.105724 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba644133-bb47-4646-8051-d76a34e08aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:51.106001 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba644133-bb47-4646-8051-d76a34e08aa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:51.106656 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba644133-bb47-4646-8051-d76a34e08aa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:24:51.107075 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba644133-bb47-4646-8051-d76a34e08aa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:51.108757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba644133-bb47-4646-8051-d76a34e08aa2 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:51.115583 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 786/1562] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:51.116176 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34840fad-ca48-4fbc-b63c-be63a2783739 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:51.124884 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 777/1563] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:24:51.125573 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba644133-bb47-4646-8051-d76a34e08aa2 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:51.127010 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34840fad-ca48-4fbc-b63c-be63a2783739 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:51.131878 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d37161-5353-4cea-bd69-06b3df5021bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:51.132590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d37161-5353-4cea-bd69-06b3df5021bc admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:51.132842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d37161-5353-4cea-bd69-06b3df5021bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:51.133119 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d37161-5353-4cea-bd69-06b3df5021bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:51.133762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d37161-5353-4cea-bd69-06b3df5021bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:51.135725 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d37161-5353-4cea-bd69-06b3df5021bc admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:51.139959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba644133-bb47-4646-8051-d76a34e08aa2 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:51.147116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187364c4-8a61-4342-a973-8dad8d926efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:51.147750 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187364c4-8a61-4342-a973-8dad8d926efa admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:51.148024 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187364c4-8a61-4342-a973-8dad8d926efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:51.148292 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187364c4-8a61-4342-a973-8dad8d926efa admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:51.148937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187364c4-8a61-4342-a973-8dad8d926efa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:24:51.149234 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187364c4-8a61-4342-a973-8dad8d926efa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:51.150846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187364c4-8a61-4342-a973-8dad8d926efa admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:51.162682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 778/1564] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:24:51.168699 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187364c4-8a61-4342-a973-8dad8d926efa admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:51.173339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187364c4-8a61-4342-a973-8dad8d926efa admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:51.177946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67274dfb-fbf2-49e3-934c-665117b1522b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:51.178535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67274dfb-fbf2-49e3-934c-665117b1522b admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:51.178863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67274dfb-fbf2-49e3-934c-665117b1522b admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:51.179152 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67274dfb-fbf2-49e3-934c-665117b1522b admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:51.179782 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67274dfb-fbf2-49e3-934c-665117b1522b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:24:51.180111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67274dfb-fbf2-49e3-934c-665117b1522b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:51.181685 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67274dfb-fbf2-49e3-934c-665117b1522b admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:51.194724 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 779/1565] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:24:51.195556 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67274dfb-fbf2-49e3-934c-665117b1522b admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:51.205423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67274dfb-fbf2-49e3-934c-665117b1522b admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:51.209928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173129fe-adef-481f-b40a-2cc020d65f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:51.210500 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173129fe-adef-481f-b40a-2cc020d65f66 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:51.210780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173129fe-adef-481f-b40a-2cc020d65f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:51.211082 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173129fe-adef-481f-b40a-2cc020d65f66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:51.211703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173129fe-adef-481f-b40a-2cc020d65f66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:24:51.212002 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173129fe-adef-481f-b40a-2cc020d65f66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:51.213553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173129fe-adef-481f-b40a-2cc020d65f66 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:51.224832 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 787/1566] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:51 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:51.224832 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d37161-5353-4cea-bd69-06b3df5021bc admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:51.229102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 780/1567] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:24:51.229102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173129fe-adef-481f-b40a-2cc020d65f66 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:51.239938 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173129fe-adef-481f-b40a-2cc020d65f66 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:51.290855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d37161-5353-4cea-bd69-06b3df5021bc admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:51.296301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee26a9d-0266-49c6-8c1d-d89a11f86686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:51.296882 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15d677-1186-4bd5-bb48-93e41c6f7f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:51.297027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee26a9d-0266-49c6-8c1d-d89a11f86686 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:51.297178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee26a9d-0266-49c6-8c1d-d89a11f86686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:51.297437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee26a9d-0266-49c6-8c1d-d89a11f86686 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:51.297553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15d677-1186-4bd5-bb48-93e41c6f7f33 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:51.297782 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15d677-1186-4bd5-bb48-93e41c6f7f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:51.298105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee26a9d-0266-49c6-8c1d-d89a11f86686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:51.298237 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15d677-1186-4bd5-bb48-93e41c6f7f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:51.298792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15d677-1186-4bd5-bb48-93e41c6f7f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:51.300036 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee26a9d-0266-49c6-8c1d-d89a11f86686 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:51.300401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15d677-1186-4bd5-bb48-93e41c6f7f33 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:51.311407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 781/1568] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:51.317494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15d677-1186-4bd5-bb48-93e41c6f7f33 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:51.321958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15d677-1186-4bd5-bb48-93e41c6f7f33 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:51.326849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6979af7-dd19-4d82-b787-f1c49d29f2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:51.327471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6979af7-dd19-4d82-b787-f1c49d29f2a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:51.327750 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6979af7-dd19-4d82-b787-f1c49d29f2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:51.328019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6979af7-dd19-4d82-b787-f1c49d29f2a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:51.328650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6979af7-dd19-4d82-b787-f1c49d29f2a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:24:51.328951 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6979af7-dd19-4d82-b787-f1c49d29f2a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:51.330661 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6979af7-dd19-4d82-b787-f1c49d29f2a8 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:51.342193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 782/1569] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:24:51.342996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6979af7-dd19-4d82-b787-f1c49d29f2a8 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:51.352937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6979af7-dd19-4d82-b787-f1c49d29f2a8 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:51.358363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461bf8d-03a8-4601-9221-1fdc9f24de25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:51.358994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461bf8d-03a8-4601-9221-1fdc9f24de25 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:51.359274 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461bf8d-03a8-4601-9221-1fdc9f24de25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:51.359546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461bf8d-03a8-4601-9221-1fdc9f24de25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:51.360167 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461bf8d-03a8-4601-9221-1fdc9f24de25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:51.362069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461bf8d-03a8-4601-9221-1fdc9f24de25 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:51.476698 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 783/1570] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:51 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:51.476698 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461bf8d-03a8-4601-9221-1fdc9f24de25 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:51.489322 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 788/1571] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:51 2022] POST /identity/v3/users => generated 289 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:51.489322 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee26a9d-0266-49c6-8c1d-d89a11f86686 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:51.496869 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee26a9d-0266-49c6-8c1d-d89a11f86686 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:51.542666 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461bf8d-03a8-4601-9221-1fdc9f24de25 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:51.548310 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd558b38-2a6f-4dfe-9569-8787cdf07bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:51.548924 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd558b38-2a6f-4dfe-9569-8787cdf07bec admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:51.549206 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd558b38-2a6f-4dfe-9569-8787cdf07bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:51.549479 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd558b38-2a6f-4dfe-9569-8787cdf07bec admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:51.550114 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd558b38-2a6f-4dfe-9569-8787cdf07bec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:51.552051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd558b38-2a6f-4dfe-9569-8787cdf07bec admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:51.555917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43708b-1a47-4e61-bbf2-589a27496672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:51.556711 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43708b-1a47-4e61-bbf2-589a27496672 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:51.556993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43708b-1a47-4e61-bbf2-589a27496672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:51.557588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43708b-1a47-4e61-bbf2-589a27496672 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:51.558227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43708b-1a47-4e61-bbf2-589a27496672 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:51.560896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43708b-1a47-4e61-bbf2-589a27496672 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:51.568297 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:51.640917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 784/1572] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:51 2022] POST /identity/v3/projects => generated 363 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:51.641924 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd558b38-2a6f-4dfe-9569-8787cdf07bec admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:51.664364 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 789/1573] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:51 2022] POST /identity/v3/projects => generated 403 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:51.664364 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43708b-1a47-4e61-bbf2-589a27496672 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:51.708928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd558b38-2a6f-4dfe-9569-8787cdf07bec admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:51.717951 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:51.728154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43708b-1a47-4e61-bbf2-589a27496672 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:51.733710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59046dcd-343f-4b65-b125-dac5d3c04a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:51.734503 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59046dcd-343f-4b65-b125-dac5d3c04a99 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:51.734812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59046dcd-343f-4b65-b125-dac5d3c04a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:51.735156 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59046dcd-343f-4b65-b125-dac5d3c04a99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:51.735948 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59046dcd-343f-4b65-b125-dac5d3c04a99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:51.737817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59046dcd-343f-4b65-b125-dac5d3c04a99 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:51.767092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7641e-e083-46dc-919b-28b0a2ae739e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:51.767728 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7641e-e083-46dc-919b-28b0a2ae739e admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:51.768010 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7641e-e083-46dc-919b-28b0a2ae739e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:51.768284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7641e-e083-46dc-919b-28b0a2ae739e admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:51.768913 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7641e-e083-46dc-919b-28b0a2ae739e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:51.770820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7641e-e083-46dc-919b-28b0a2ae739e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:51.873926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 790/1574] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:51 2022] POST /identity/v3/projects => generated 389 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:51.874975 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59046dcd-343f-4b65-b125-dac5d3c04a99 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:51.884091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 785/1575] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:51 2022] POST /identity/v3/users => generated 292 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:51.886955 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7641e-e083-46dc-919b-28b0a2ae739e admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:51.896780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7641e-e083-46dc-919b-28b0a2ae739e admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:51.941014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59046dcd-343f-4b65-b125-dac5d3c04a99 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:51.945762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcaa85e-3050-49bc-962d-3a848d4cac4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:51.946563 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcaa85e-3050-49bc-962d-3a848d4cac4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:51.946849 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcaa85e-3050-49bc-962d-3a848d4cac4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:51.947160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcaa85e-3050-49bc-962d-3a848d4cac4d admin admin] PATH_INFO: `/v3/projects/7239e9b25901478398e787e38e51725f/users/793b30c8c0df4e789e745fe4a1a10eea/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:51.958706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876bf0c-10db-42d6-a97a-3ecbb2ce8e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:51.959359 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876bf0c-10db-42d6-a97a-3ecbb2ce8e4d admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:51.959641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876bf0c-10db-42d6-a97a-3ecbb2ce8e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:51.959920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876bf0c-10db-42d6-a97a-3ecbb2ce8e4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:51.960581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876bf0c-10db-42d6-a97a-3ecbb2ce8e4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:51.962649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876bf0c-10db-42d6-a97a-3ecbb2ce8e4d admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:51.962996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcaa85e-3050-49bc-962d-3a848d4cac4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7239e9b25901478398e787e38e51725f, user_id=793b30c8c0df4e789e745fe4a1a10eea, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:51.964031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: WARNING py.warnings [None req-6dcaa85e-3050-49bc-962d-3a848d4cac4d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:24:51.964031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: warnings.warn(msg) Aug 31 19:24:51.964031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: Aug 31 19:24:51.966948 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcaa85e-3050-49bc-962d-3a848d4cac4d admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.027779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 791/1576] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:51 2022] PUT /identity/v3/projects/7239e9b25901478398e787e38e51725f/users/793b30c8c0df4e789e745fe4a1a10eea/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:52.029518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcaa85e-3050-49bc-962d-3a848d4cac4d admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.077302 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 786/1577] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:51 2022] POST /identity/v3/projects => generated 391 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:52.077302 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876bf0c-10db-42d6-a97a-3ecbb2ce8e4d admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.095535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:52.098113 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcaa85e-3050-49bc-962d-3a848d4cac4d admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.139039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876bf0c-10db-42d6-a97a-3ecbb2ce8e4d admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.143555 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755db381-07e9-4626-928b-02659df019b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.144157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755db381-07e9-4626-928b-02659df019b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.144436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755db381-07e9-4626-928b-02659df019b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.144710 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755db381-07e9-4626-928b-02659df019b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.145341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755db381-07e9-4626-928b-02659df019b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.146969 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755db381-07e9-4626-928b-02659df019b4 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.156049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11395f7a-a21e-4968-9bd0-2d771a817eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.156687 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11395f7a-a21e-4968-9bd0-2d771a817eeb admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.157061 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11395f7a-a21e-4968-9bd0-2d771a817eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.157362 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11395f7a-a21e-4968-9bd0-2d771a817eeb admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.158002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11395f7a-a21e-4968-9bd0-2d771a817eeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.158857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 787/1578] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:52.162313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11395f7a-a21e-4968-9bd0-2d771a817eeb admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.162433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755db381-07e9-4626-928b-02659df019b4 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.172004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755db381-07e9-4626-928b-02659df019b4 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.176983 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7133bf8-6dc6-4d21-b612-a6ed3ffbd52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.177582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7133bf8-6dc6-4d21-b612-a6ed3ffbd52b admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.177877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7133bf8-6dc6-4d21-b612-a6ed3ffbd52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.178132 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7133bf8-6dc6-4d21-b612-a6ed3ffbd52b admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.178763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7133bf8-6dc6-4d21-b612-a6ed3ffbd52b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.180709 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7133bf8-6dc6-4d21-b612-a6ed3ffbd52b admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.267689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:52.277073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 792/1579] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:52 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:52.277073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11395f7a-a21e-4968-9bd0-2d771a817eeb admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.277905 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 788/1580] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:52 2022] POST /identity/v3/users => generated 282 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:52.277905 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7133bf8-6dc6-4d21-b612-a6ed3ffbd52b admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.285953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11395f7a-a21e-4968-9bd0-2d771a817eeb admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.287933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7133bf8-6dc6-4d21-b612-a6ed3ffbd52b admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.291026 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a2a43-bd53-4cc8-b862-ddfc3dd57027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.291724 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a2a43-bd53-4cc8-b862-ddfc3dd57027 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.292008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a2a43-bd53-4cc8-b862-ddfc3dd57027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.292283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a2a43-bd53-4cc8-b862-ddfc3dd57027 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.292668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2a470-0328-4bbe-8977-6675791c51b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.292920 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a2a43-bd53-4cc8-b862-ddfc3dd57027 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.293395 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2a470-0328-4bbe-8977-6675791c51b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.293661 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2a470-0328-4bbe-8977-6675791c51b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.293933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2a470-0328-4bbe-8977-6675791c51b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.295455 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a2a43-bd53-4cc8-b862-ddfc3dd57027 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.296122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2a470-0328-4bbe-8977-6675791c51b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.298212 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2a470-0328-4bbe-8977-6675791c51b1 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.386144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 793/1581] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:52 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:52.387189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a2a43-bd53-4cc8-b862-ddfc3dd57027 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.396679 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a2a43-bd53-4cc8-b862-ddfc3dd57027 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.401213 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eddb14-46ca-4ee7-973d-82a10ba8122b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.401794 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eddb14-46ca-4ee7-973d-82a10ba8122b admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.402089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eddb14-46ca-4ee7-973d-82a10ba8122b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.402344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eddb14-46ca-4ee7-973d-82a10ba8122b admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.402994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eddb14-46ca-4ee7-973d-82a10ba8122b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.405086 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eddb14-46ca-4ee7-973d-82a10ba8122b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.415505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 794/1582] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:52.416798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83e1f1-75be-40ba-b1fd-611d7c1ec1a6 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.417083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83e1f1-75be-40ba-b1fd-611d7c1ec1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.417465 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83e1f1-75be-40ba-b1fd-611d7c1ec1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.438436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 789/1583] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:52 2022] POST /identity/v3/users => generated 295 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:52.439361 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2a470-0328-4bbe-8977-6675791c51b1 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.448142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-db83e1f1-75be-40ba-b1fd-611d7c1ec1a6 None None] MFA Rules not processed for user `793b30c8c0df4e789e745fe4a1a10eea`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:52.449098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2a470-0328-4bbe-8977-6675791c51b1 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.449344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-db83e1f1-75be-40ba-b1fd-611d7c1ec1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:52.454276 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e9381-f0c0-40df-80ec-e7f8463e3119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.454900 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e9381-f0c0-40df-80ec-e7f8463e3119 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.455194 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e9381-f0c0-40df-80ec-e7f8463e3119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.455470 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e9381-f0c0-40df-80ec-e7f8463e3119 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.456122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e9381-f0c0-40df-80ec-e7f8463e3119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.458061 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e9381-f0c0-40df-80ec-e7f8463e3119 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.513653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 795/1584] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:52.514712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83e1f1-75be-40ba-b1fd-611d7c1ec1a6 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.526591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83e1f1-75be-40ba-b1fd-611d7c1ec1a6 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.531977 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e33c4-6d2d-48de-b75d-d66adc414c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.532917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e33c4-6d2d-48de-b75d-d66adc414c74 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.532917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e33c4-6d2d-48de-b75d-d66adc414c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.533110 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e33c4-6d2d-48de-b75d-d66adc414c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.533734 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e33c4-6d2d-48de-b75d-d66adc414c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.535346 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e33c4-6d2d-48de-b75d-d66adc414c74 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.549898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 796/1585] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:52.549898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e33c4-6d2d-48de-b75d-d66adc414c74 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.559396 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e33c4-6d2d-48de-b75d-d66adc414c74 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.564773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0c0b5-05be-42c0-9861-512aab4ba463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.564949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 790/1586] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:52 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:52.568159 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0c0b5-05be-42c0-9861-512aab4ba463 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.568269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e9381-f0c0-40df-80ec-e7f8463e3119 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.568377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0c0b5-05be-42c0-9861-512aab4ba463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.568377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0c0b5-05be-42c0-9861-512aab4ba463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.568377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0c0b5-05be-42c0-9861-512aab4ba463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.568625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0c0b5-05be-42c0-9861-512aab4ba463 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.575165 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e9381-f0c0-40df-80ec-e7f8463e3119 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.582625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 797/1587] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:52.582625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0c0b5-05be-42c0-9861-512aab4ba463 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.582852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb52172-4102-40ce-ae1e-9208ba89fd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.582852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb52172-4102-40ce-ae1e-9208ba89fd3e admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.582852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb52172-4102-40ce-ae1e-9208ba89fd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.582852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb52172-4102-40ce-ae1e-9208ba89fd3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.583271 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb52172-4102-40ce-ae1e-9208ba89fd3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.584869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb52172-4102-40ce-ae1e-9208ba89fd3e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.589990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0c0b5-05be-42c0-9861-512aab4ba463 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.595319 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ecbb7-9a11-4999-9e8f-b66c90922532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.600838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ecbb7-9a11-4999-9e8f-b66c90922532 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.600973 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 791/1588] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:52.601076 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ecbb7-9a11-4999-9e8f-b66c90922532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.601076 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ecbb7-9a11-4999-9e8f-b66c90922532 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.601245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb52172-4102-40ce-ae1e-9208ba89fd3e admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.601346 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ecbb7-9a11-4999-9e8f-b66c90922532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.601346 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ecbb7-9a11-4999-9e8f-b66c90922532 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.608229 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb52172-4102-40ce-ae1e-9208ba89fd3e admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.612230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 798/1589] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:52.613055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ecbb7-9a11-4999-9e8f-b66c90922532 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.616752 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f7d50-6b5d-4aaf-a4ed-5d955bb1c067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.617618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f7d50-6b5d-4aaf-a4ed-5d955bb1c067 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.617938 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f7d50-6b5d-4aaf-a4ed-5d955bb1c067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.618839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f7d50-6b5d-4aaf-a4ed-5d955bb1c067 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.622003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f7d50-6b5d-4aaf-a4ed-5d955bb1c067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.622003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f7d50-6b5d-4aaf-a4ed-5d955bb1c067 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.623329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ecbb7-9a11-4999-9e8f-b66c90922532 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.627944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b6da5-0563-4dcb-b0b8-6bf1837c320e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.628542 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b6da5-0563-4dcb-b0b8-6bf1837c320e admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.628817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b6da5-0563-4dcb-b0b8-6bf1837c320e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.629085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b6da5-0563-4dcb-b0b8-6bf1837c320e admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.629724 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b6da5-0563-4dcb-b0b8-6bf1837c320e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.631356 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b6da5-0563-4dcb-b0b8-6bf1837c320e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.632032 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 792/1590] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:52.633873 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795bdc0-8a94-43df-87ce-debf39af449d None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.634156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795bdc0-8a94-43df-87ce-debf39af449d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.634430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795bdc0-8a94-43df-87ce-debf39af449d None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.635951 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 793/1591] 10.222.0.59 () {54 vars in 962 bytes} [Wed Aug 31 19:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:24:52.636566 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795bdc0-8a94-43df-87ce-debf39af449d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.642096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 799/1592] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:52.642917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b6da5-0563-4dcb-b0b8-6bf1837c320e admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.654058 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795bdc0-8a94-43df-87ce-debf39af449d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.654249 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b6da5-0563-4dcb-b0b8-6bf1837c320e admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.656553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e36aab4-bb30-4e50-9236-ec62ffe1c71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.657127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c17f8b-e81d-4bd9-9be8-23de9d6683bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.657254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36aab4-bb30-4e50-9236-ec62ffe1c71f admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.657472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36aab4-bb30-4e50-9236-ec62ffe1c71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.657722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c17f8b-e81d-4bd9-9be8-23de9d6683bc admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.657863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36aab4-bb30-4e50-9236-ec62ffe1c71f admin admin] PATH_INFO: `/v3/projects/fa8bb54679c74f039eee057c2b4a84f7/users/7aaa082f2b754cf5aa941586361c1877/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.658027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c17f8b-e81d-4bd9-9be8-23de9d6683bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.658261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c17f8b-e81d-4bd9-9be8-23de9d6683bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.658921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c17f8b-e81d-4bd9-9be8-23de9d6683bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.660496 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c17f8b-e81d-4bd9-9be8-23de9d6683bc admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.671063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 800/1593] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:52.671871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c17f8b-e81d-4bd9-9be8-23de9d6683bc admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.677451 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e36aab4-bb30-4e50-9236-ec62ffe1c71f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa8bb54679c74f039eee057c2b4a84f7, user_id=7aaa082f2b754cf5aa941586361c1877, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.680148 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e36aab4-bb30-4e50-9236-ec62ffe1c71f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.681556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c17f8b-e81d-4bd9-9be8-23de9d6683bc admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.686136 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d5973-2a98-4420-a0ea-38a1c66c5ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.686723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d5973-2a98-4420-a0ea-38a1c66c5ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.687138 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d5973-2a98-4420-a0ea-38a1c66c5ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.687412 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d5973-2a98-4420-a0ea-38a1c66c5ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.688037 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d5973-2a98-4420-a0ea-38a1c66c5ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.689606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d5973-2a98-4420-a0ea-38a1c66c5ef1 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.700042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 801/1594] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:52.702153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d5973-2a98-4420-a0ea-38a1c66c5ef1 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.710578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d5973-2a98-4420-a0ea-38a1c66c5ef1 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.715135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa636fa-8d6f-4057-a49d-b4539628a61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.715698 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa636fa-8d6f-4057-a49d-b4539628a61b admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.716064 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa636fa-8d6f-4057-a49d-b4539628a61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.716894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa636fa-8d6f-4057-a49d-b4539628a61b admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.717553 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa636fa-8d6f-4057-a49d-b4539628a61b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.719146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa636fa-8d6f-4057-a49d-b4539628a61b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.729591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 802/1595] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:52.730372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa636fa-8d6f-4057-a49d-b4539628a61b admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.739841 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa636fa-8d6f-4057-a49d-b4539628a61b admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.744421 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014125a-db81-4ec1-b890-740b7535c0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.745008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014125a-db81-4ec1-b890-740b7535c0ab admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.745286 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014125a-db81-4ec1-b890-740b7535c0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.745561 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014125a-db81-4ec1-b890-740b7535c0ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.746191 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014125a-db81-4ec1-b890-740b7535c0ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.747194 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 794/1596] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:52 2022] PUT /identity/v3/projects/fa8bb54679c74f039eee057c2b4a84f7/users/7aaa082f2b754cf5aa941586361c1877/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:52.750913 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e36aab4-bb30-4e50-9236-ec62ffe1c71f admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.751060 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014125a-db81-4ec1-b890-740b7535c0ab admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.759218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 803/1597] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:52.760376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863aa46-ab33-48c2-b525-e95ab4030bb7 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.760665 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863aa46-ab33-48c2-b525-e95ab4030bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.762208 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863aa46-ab33-48c2-b525-e95ab4030bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.800263 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-e863aa46-ab33-48c2-b525-e95ab4030bb7 None None] MFA Rules not processed for user `d461422ce98442bcbe6b254a8dad5e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:52.801475 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e863aa46-ab33-48c2-b525-e95ab4030bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:52.813854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:52.815297 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e36aab4-bb30-4e50-9236-ec62ffe1c71f admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.820542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97604ada-9ef7-4076-be99-9cfa64313ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.821208 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97604ada-9ef7-4076-be99-9cfa64313ce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.821494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97604ada-9ef7-4076-be99-9cfa64313ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.821783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97604ada-9ef7-4076-be99-9cfa64313ce8 admin admin] PATH_INFO: `/v3/projects/b035a36532cd45689246f990030a3a76/users/6b94b7297d0f4676a6272ea0c6e99895/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.839126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97604ada-9ef7-4076-be99-9cfa64313ce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b035a36532cd45689246f990030a3a76, user_id=6b94b7297d0f4676a6272ea0c6e99895, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.842432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97604ada-9ef7-4076-be99-9cfa64313ce8 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.870036 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 804/1598] 10.222.0.59 () {58 vars in 1059 bytes} [Wed Aug 31 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:52.870896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863aa46-ab33-48c2-b525-e95ab4030bb7 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.880891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863aa46-ab33-48c2-b525-e95ab4030bb7 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.885624 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40045569-2683-4baa-baab-72756108edeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.886207 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40045569-2683-4baa-baab-72756108edeb admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.886489 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40045569-2683-4baa-baab-72756108edeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.886761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40045569-2683-4baa-baab-72756108edeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.887428 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40045569-2683-4baa-baab-72756108edeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.889795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40045569-2683-4baa-baab-72756108edeb admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.900038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 805/1599] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:52.900854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40045569-2683-4baa-baab-72756108edeb admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.911472 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40045569-2683-4baa-baab-72756108edeb admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.915913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a5c74-ce1c-4e0d-b42b-bbc8894b0779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:52.916472 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a5c74-ce1c-4e0d-b42b-bbc8894b0779 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:52.916788 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a5c74-ce1c-4e0d-b42b-bbc8894b0779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:52.917060 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a5c74-ce1c-4e0d-b42b-bbc8894b0779 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:52.917735 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a5c74-ce1c-4e0d-b42b-bbc8894b0779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:52.919162 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:52.919403 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a5c74-ce1c-4e0d-b42b-bbc8894b0779 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:52.924948 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 795/1600] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:52 2022] PUT /identity/v3/projects/b035a36532cd45689246f990030a3a76/users/6b94b7297d0f4676a6272ea0c6e99895/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:52.935200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97604ada-9ef7-4076-be99-9cfa64313ce8 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.937855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 806/1601] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:52.937855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a5c74-ce1c-4e0d-b42b-bbc8894b0779 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:52.942189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a5c74-ce1c-4e0d-b42b-bbc8894b0779 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:52.997232 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97604ada-9ef7-4076-be99-9cfa64313ce8 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:53.000741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7d6e6-b3f7-4131-9a59-1b7ff20d1a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:53.001409 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7d6e6-b3f7-4131-9a59-1b7ff20d1a7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.001689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7d6e6-b3f7-4131-9a59-1b7ff20d1a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.001959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7d6e6-b3f7-4131-9a59-1b7ff20d1a7e admin admin] PATH_INFO: `/v3/projects/30121138fee54489bdebebd5345abac5/users/bbb071487d364e708c8ba6e1bb2de8d6/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.003225 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186248bc-941a-4468-8d25-e7ecf1c9118a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:53.003891 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186248bc-941a-4468-8d25-e7ecf1c9118a admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.004174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186248bc-941a-4468-8d25-e7ecf1c9118a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.004450 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186248bc-941a-4468-8d25-e7ecf1c9118a admin admin] PATH_INFO: `/v3/projects/a13c903a376b46508e8cccbcb20b484c/users/8bc4801f1f674f55b57d49e5f09205ea/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.017457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7d6e6-b3f7-4131-9a59-1b7ff20d1a7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=30121138fee54489bdebebd5345abac5, user_id=bbb071487d364e708c8ba6e1bb2de8d6, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:53.019963 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186248bc-941a-4468-8d25-e7ecf1c9118a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a13c903a376b46508e8cccbcb20b484c, user_id=8bc4801f1f674f55b57d49e5f09205ea, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:53.027672 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7d6e6-b3f7-4131-9a59-1b7ff20d1a7e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:53.027791 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186248bc-941a-4468-8d25-e7ecf1c9118a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:53.142451 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 807/1602] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:52 2022] PUT /identity/v3/projects/30121138fee54489bdebebd5345abac5/users/bbb071487d364e708c8ba6e1bb2de8d6/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:53.143795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd572df-bd34-4230-a7f0-2bb000d5a7c0 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.144063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd572df-bd34-4230-a7f0-2bb000d5a7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.144336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd572df-bd34-4230-a7f0-2bb000d5a7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.158468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 796/1603] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:52 2022] PUT /identity/v3/projects/a13c903a376b46508e8cccbcb20b484c/users/8bc4801f1f674f55b57d49e5f09205ea/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:53.159424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186248bc-941a-4468-8d25-e7ecf1c9118a admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:53.163854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-3fd572df-bd34-4230-a7f0-2bb000d5a7c0 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:53.164992 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3fd572df-bd34-4230-a7f0-2bb000d5a7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:53.230419 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186248bc-941a-4468-8d25-e7ecf1c9118a admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:53.231024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 808/1604] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:53.232034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd572df-bd34-4230-a7f0-2bb000d5a7c0 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:53.242558 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e557a8-6edb-4c69-8413-cff5a06eda92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:53.250327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e557a8-6edb-4c69-8413-cff5a06eda92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.250327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e557a8-6edb-4c69-8413-cff5a06eda92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.250327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e557a8-6edb-4c69-8413-cff5a06eda92 admin admin] PATH_INFO: `/v3/projects/a5258453bce34fb680a34963e4bbe733/users/ef8c73aa3c7e4adca83cd151fd88fae1/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.250646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:53.259488 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e557a8-6edb-4c69-8413-cff5a06eda92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5258453bce34fb680a34963e4bbe733, user_id=ef8c73aa3c7e4adca83cd151fd88fae1, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:53.262428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e557a8-6edb-4c69-8413-cff5a06eda92 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:53.304909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd572df-bd34-4230-a7f0-2bb000d5a7c0 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:53.310327 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad6d59-acd0-4aed-8227-f68ae3a0dae1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:53.311601 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad6d59-acd0-4aed-8227-f68ae3a0dae1 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.311884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad6d59-acd0-4aed-8227-f68ae3a0dae1 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.312157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad6d59-acd0-4aed-8227-f68ae3a0dae1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.360673 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 797/1605] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:53 2022] PUT /identity/v3/projects/a5258453bce34fb680a34963e4bbe733/users/ef8c73aa3c7e4adca83cd151fd88fae1/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:53.360673 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e557a8-6edb-4c69-8413-cff5a06eda92 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:53.376587 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad6d59-acd0-4aed-8227-f68ae3a0dae1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:53.378246 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad6d59-acd0-4aed-8227-f68ae3a0dae1 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:53.387438 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 809/1606] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:53.388774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc7195-18ad-4612-b669-7d1f25ba4ca5 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.389779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc7195-18ad-4612-b669-7d1f25ba4ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.390197 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc7195-18ad-4612-b669-7d1f25ba4ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.420797 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-41fc7195-18ad-4612-b669-7d1f25ba4ca5 None None] MFA Rules not processed for user `6b94b7297d0f4676a6272ea0c6e99895`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:53.421997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-41fc7195-18ad-4612-b669-7d1f25ba4ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:53.430387 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e557a8-6edb-4c69-8413-cff5a06eda92 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:53.438171 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e82cf-c6bb-41eb-80a4-a9c61a2ac8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:53.439050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e82cf-c6bb-41eb-80a4-a9c61a2ac8be admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.439649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e82cf-c6bb-41eb-80a4-a9c61a2ac8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.439751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e82cf-c6bb-41eb-80a4-a9c61a2ac8be admin admin] PATH_INFO: `/v3/projects/4b5588a49a314579a29bad0f73aae8b4/users/0d5336bcea514f378e06737680ae9bdc/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.449755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:53.460516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e82cf-c6bb-41eb-80a4-a9c61a2ac8be admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b5588a49a314579a29bad0f73aae8b4, user_id=0d5336bcea514f378e06737680ae9bdc, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:53.464334 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e82cf-c6bb-41eb-80a4-a9c61a2ac8be admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:53.489697 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 810/1607] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:53.490782 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc7195-18ad-4612-b669-7d1f25ba4ca5 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:53.504953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc7195-18ad-4612-b669-7d1f25ba4ca5 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:53.510037 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd662a7-d85e-498e-b19a-06c58c7a4a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:53.516556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd662a7-d85e-498e-b19a-06c58c7a4a02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.516556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd662a7-d85e-498e-b19a-06c58c7a4a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.516556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd662a7-d85e-498e-b19a-06c58c7a4a02 admin admin] PATH_INFO: `/v3/projects/a7e6027f7c1241dcb09786ff94df5c2f/users/d441bbdb43c5403abf8480146bc15753/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.528918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd662a7-d85e-498e-b19a-06c58c7a4a02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7e6027f7c1241dcb09786ff94df5c2f, user_id=d441bbdb43c5403abf8480146bc15753, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:53.531657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd662a7-d85e-498e-b19a-06c58c7a4a02 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:53.542058 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 798/1608] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:53 2022] PUT /identity/v3/projects/4b5588a49a314579a29bad0f73aae8b4/users/0d5336bcea514f378e06737680ae9bdc/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:53.545154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df06a93-6639-4bb0-b228-827e62c23fae None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.545154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df06a93-6639-4bb0-b228-827e62c23fae None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.545154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df06a93-6639-4bb0-b228-827e62c23fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.563662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-5df06a93-6639-4bb0-b228-827e62c23fae None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:53.564856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-5df06a93-6639-4bb0-b228-827e62c23fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:53.599559 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 811/1609] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:53 2022] PUT /identity/v3/projects/a7e6027f7c1241dcb09786ff94df5c2f/users/d441bbdb43c5403abf8480146bc15753/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:53.600958 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84b248-b793-428a-9679-07dea08ae655 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.601338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84b248-b793-428a-9679-07dea08ae655 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.601851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84b248-b793-428a-9679-07dea08ae655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.632757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 799/1610] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:53.633428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df06a93-6639-4bb0-b228-827e62c23fae None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:53.637615 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:53.641635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-5c84b248-b793-428a-9679-07dea08ae655 None None] MFA Rules not processed for user `bbb071487d364e708c8ba6e1bb2de8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:53.643109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-5c84b248-b793-428a-9679-07dea08ae655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:53.691669 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df06a93-6639-4bb0-b228-827e62c23fae None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:53.710244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 812/1611] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:53.711695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebdd8a-8235-4c5e-8f14-78c7a131bf27 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.711957 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebdd8a-8235-4c5e-8f14-78c7a131bf27 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.712229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebdd8a-8235-4c5e-8f14-78c7a131bf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.741349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-76ebdd8a-8235-4c5e-8f14-78c7a131bf27 None None] MFA Rules not processed for user `ef8c73aa3c7e4adca83cd151fd88fae1`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:53.742596 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-76ebdd8a-8235-4c5e-8f14-78c7a131bf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:53.745576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ec6cd-eefd-40f4-a9ee-e86899d864a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:53.746190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ec6cd-eefd-40f4-a9ee-e86899d864a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.746473 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ec6cd-eefd-40f4-a9ee-e86899d864a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.746762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ec6cd-eefd-40f4-a9ee-e86899d864a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.807215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:53.810110 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 813/1612] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:53 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:53.811400 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ec6cd-eefd-40f4-a9ee-e86899d864a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:53.812363 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d55f27-0dc8-42f8-a23b-92af3199d18b None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.815664 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d55f27-0dc8-42f8-a23b-92af3199d18b None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.815664 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d55f27-0dc8-42f8-a23b-92af3199d18b None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.815901 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ec6cd-eefd-40f4-a9ee-e86899d864a0 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:53.816003 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 814/1613] 10.222.0.59 () {54 vars in 962 bytes} [Wed Aug 31 19:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:24:53.816003 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0114a88-51f9-4f2a-a331-d37d69639e96 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.816003 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0114a88-51f9-4f2a-a331-d37d69639e96 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.816003 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0114a88-51f9-4f2a-a331-d37d69639e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.825193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 800/1614] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:53.827370 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484e19c-e800-42a0-b0fb-9e1350bdc4ca None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.827663 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484e19c-e800-42a0-b0fb-9e1350bdc4ca None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.827970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484e19c-e800-42a0-b0fb-9e1350bdc4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.840348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-f0114a88-51f9-4f2a-a331-d37d69639e96 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:53.841550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f0114a88-51f9-4f2a-a331-d37d69639e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:53.849718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-2484e19c-e800-42a0-b0fb-9e1350bdc4ca None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:53.850324 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 815/1615] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:53.851095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-2484e19c-e800-42a0-b0fb-9e1350bdc4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:53.851588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f603e-5135-490e-a977-6e9a4cb7596a None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.851866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f603e-5135-490e-a977-6e9a4cb7596a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.852392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f603e-5135-490e-a977-6e9a4cb7596a None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.853878 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 816/1616] 10.222.0.59 () {54 vars in 912 bytes} [Wed Aug 31 19:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:24:53.854834 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f603e-5135-490e-a977-6e9a4cb7596a None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:53.864930 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 801/1617] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:53.866015 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484e19c-e800-42a0-b0fb-9e1350bdc4ca None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:53.866158 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f603e-5135-490e-a977-6e9a4cb7596a None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:53.869845 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184fb0a-ebd3-43fe-8f1c-a8d7bda1bd5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:53.870438 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184fb0a-ebd3-43fe-8f1c-a8d7bda1bd5f service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.871063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184fb0a-ebd3-43fe-8f1c-a8d7bda1bd5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.871063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184fb0a-ebd3-43fe-8f1c-a8d7bda1bd5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.877469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484e19c-e800-42a0-b0fb-9e1350bdc4ca None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:53.882409 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2744c89-fdce-405f-b6f5-eba909f408ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:53.882763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184fb0a-ebd3-43fe-8f1c-a8d7bda1bd5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:53.883035 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2744c89-fdce-405f-b6f5-eba909f408ce admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.883312 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2744c89-fdce-405f-b6f5-eba909f408ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.883580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2744c89-fdce-405f-b6f5-eba909f408ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.884204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2744c89-fdce-405f-b6f5-eba909f408ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:53.884323 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184fb0a-ebd3-43fe-8f1c-a8d7bda1bd5f service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:53.886075 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2744c89-fdce-405f-b6f5-eba909f408ce admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:53.894552 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 817/1618] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:53.895365 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184fb0a-ebd3-43fe-8f1c-a8d7bda1bd5f service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:53.905002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184fb0a-ebd3-43fe-8f1c-a8d7bda1bd5f service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:53.909603 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3dd0b-5de1-45db-83bc-3167b9566a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:53.910219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3dd0b-5de1-45db-83bc-3167b9566a36 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.910505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3dd0b-5de1-45db-83bc-3167b9566a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.910768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3dd0b-5de1-45db-83bc-3167b9566a36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.911481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e3dd0b-5de1-45db-83bc-3167b9566a36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:53.913367 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e3dd0b-5de1-45db-83bc-3167b9566a36 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:53.956760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 802/1619] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:53 2022] POST /identity/v3/projects => generated 365 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:53.956760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8a303-26b5-426c-b715-b7cc8b4ffda7 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:53.956760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8a303-26b5-426c-b715-b7cc8b4ffda7 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:53.956760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8a303-26b5-426c-b715-b7cc8b4ffda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:53.977942 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-59a8a303-26b5-426c-b715-b7cc8b4ffda7 None None] MFA Rules not processed for user `d461422ce98442bcbe6b254a8dad5e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:53.982098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-59a8a303-26b5-426c-b715-b7cc8b4ffda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:54.024165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 818/1620] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:53 2022] POST /identity/v3/projects => generated 405 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:54.025727 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29add1de-825f-486a-8899-f64b23da05ac None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:54.026241 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29add1de-825f-486a-8899-f64b23da05ac None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:54.026516 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29add1de-825f-486a-8899-f64b23da05ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:54.048973 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-29add1de-825f-486a-8899-f64b23da05ac None None] MFA Rules not processed for user `d461422ce98442bcbe6b254a8dad5e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:54.050265 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-29add1de-825f-486a-8899-f64b23da05ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:54.057842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 803/1621] 10.222.0.59 () {58 vars in 1059 bytes} [Wed Aug 31 19:24:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:54.058607 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8a303-26b5-426c-b715-b7cc8b4ffda7 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:54.060218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-59a8a303-26b5-426c-b715-b7cc8b4ffda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:54.107009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:54.125931 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8a303-26b5-426c-b715-b7cc8b4ffda7 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:54.133463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 819/1622] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 31 19:24:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:54.134136 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29add1de-825f-486a-8899-f64b23da05ac None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:54.199449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b126382-faab-4a57-afbe-48d291406042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:54.200177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b126382-faab-4a57-afbe-48d291406042 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:54.200299 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29add1de-825f-486a-8899-f64b23da05ac None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:54.200535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b126382-faab-4a57-afbe-48d291406042 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:54.200868 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b126382-faab-4a57-afbe-48d291406042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:54.205209 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0905dbc-b57d-4e9e-9a7f-5d9ed2f0067a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:54.205834 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0905dbc-b57d-4e9e-9a7f-5d9ed2f0067a service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:54.206128 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0905dbc-b57d-4e9e-9a7f-5d9ed2f0067a service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:54.206480 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0905dbc-b57d-4e9e-9a7f-5d9ed2f0067a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:54.245799 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:54.289370 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0905dbc-b57d-4e9e-9a7f-5d9ed2f0067a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:54.290836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0905dbc-b57d-4e9e-9a7f-5d9ed2f0067a service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:54.296645 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b126382-faab-4a57-afbe-48d291406042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:54.298150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b126382-faab-4a57-afbe-48d291406042 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:54.302539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 820/1623] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:54.304123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0905dbc-b57d-4e9e-9a7f-5d9ed2f0067a service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:54.313409 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 804/1624] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:54.313409 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b126382-faab-4a57-afbe-48d291406042 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:54.313810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0905dbc-b57d-4e9e-9a7f-5d9ed2f0067a service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:54.319578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1695a1-4205-4525-9197-9520d9d8cedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:54.319750 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1695a1-4205-4525-9197-9520d9d8cedd admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:54.320015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1695a1-4205-4525-9197-9520d9d8cedd admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:54.320288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1695a1-4205-4525-9197-9520d9d8cedd admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:54.320929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1695a1-4205-4525-9197-9520d9d8cedd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:54.322814 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1695a1-4205-4525-9197-9520d9d8cedd admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:54.324207 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b126382-faab-4a57-afbe-48d291406042 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:54.332592 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902d220-d142-4d9f-9628-565d4974e387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:54.333311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902d220-d142-4d9f-9628-565d4974e387 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:54.333600 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902d220-d142-4d9f-9628-565d4974e387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:54.334027 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902d220-d142-4d9f-9628-565d4974e387 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:54.334753 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902d220-d142-4d9f-9628-565d4974e387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:54.337185 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902d220-d142-4d9f-9628-565d4974e387 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:54.391833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 821/1625] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:54 2022] POST /identity/v3/users => generated 289 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:54.391833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1695a1-4205-4525-9197-9520d9d8cedd admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:54.402216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1695a1-4205-4525-9197-9520d9d8cedd admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:54.408657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0fcde-56b1-4521-ae2c-96c6eb6c9fe3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:54.409264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0fcde-56b1-4521-ae2c-96c6eb6c9fe3 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:54.409542 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0fcde-56b1-4521-ae2c-96c6eb6c9fe3 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:54.409816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0fcde-56b1-4521-ae2c-96c6eb6c9fe3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:54.449696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 805/1626] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:54 2022] POST /identity/v3/users => generated 309 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:54.451480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902d220-d142-4d9f-9628-565d4974e387 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:54.479587 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0fcde-56b1-4521-ae2c-96c6eb6c9fe3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:54.481310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0fcde-56b1-4521-ae2c-96c6eb6c9fe3 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:54.495120 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 822/1627] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:54.508885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0fcde-56b1-4521-ae2c-96c6eb6c9fe3 service nova] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:54.524757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0fcde-56b1-4521-ae2c-96c6eb6c9fe3 service nova] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:54.530606 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902d220-d142-4d9f-9628-565d4974e387 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:54.534764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbd8fe-bca4-4061-86e9-bf875a64a6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:54.534764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbd8fe-bca4-4061-86e9-bf875a64a6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:54.534764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbd8fe-bca4-4061-86e9-bf875a64a6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:54.534764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbd8fe-bca4-4061-86e9-bf875a64a6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:54.535191 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbd8fe-bca4-4061-86e9-bf875a64a6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:54.535442 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbd8fe-bca4-4061-86e9-bf875a64a6d0 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:54.536542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2088203-4e39-4cbf-a31e-c7dab87f57cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:54.537753 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2088203-4e39-4cbf-a31e-c7dab87f57cb service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:54.538173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2088203-4e39-4cbf-a31e-c7dab87f57cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:54.538552 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2088203-4e39-4cbf-a31e-c7dab87f57cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:54.550083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 823/1628] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:54.551289 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbd8fe-bca4-4061-86e9-bf875a64a6d0 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:54.557192 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:54.558408 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2088203-4e39-4cbf-a31e-c7dab87f57cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:54.560275 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2088203-4e39-4cbf-a31e-c7dab87f57cb service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:54.563561 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbd8fe-bca4-4061-86e9-bf875a64a6d0 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:54.568885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4babb-a790-4246-8442-ebe218b3ed93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:54.574994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4babb-a790-4246-8442-ebe218b3ed93 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:54.574994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4babb-a790-4246-8442-ebe218b3ed93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:54.574994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4babb-a790-4246-8442-ebe218b3ed93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:54.574994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4babb-a790-4246-8442-ebe218b3ed93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:54.574994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4babb-a790-4246-8442-ebe218b3ed93 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:54.575418 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 806/1629] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:54.588056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 824/1630] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:54.593186 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2088203-4e39-4cbf-a31e-c7dab87f57cb service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:54.593905 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4babb-a790-4246-8442-ebe218b3ed93 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:54.614546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4babb-a790-4246-8442-ebe218b3ed93 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:54.617072 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2088203-4e39-4cbf-a31e-c7dab87f57cb service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:54.625237 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69372cc5-7b00-42d9-aa99-54e18831aec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:54.627028 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69372cc5-7b00-42d9-aa99-54e18831aec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:54.627458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69372cc5-7b00-42d9-aa99-54e18831aec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:54.628192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b258f-bf7a-4f33-8362-30d9f60d623a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:54.628883 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b258f-bf7a-4f33-8362-30d9f60d623a admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:54.629440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b258f-bf7a-4f33-8362-30d9f60d623a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:54.629702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b258f-bf7a-4f33-8362-30d9f60d623a admin admin] PATH_INFO: `/v3/projects/43bd55a6e88b4038a0772445761e7135/users/fd3c2ae863ad4c54aeb2c652d6f7e90f/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:54.630809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69372cc5-7b00-42d9-aa99-54e18831aec4 admin admin] PATH_INFO: `/v3/projects/f663d2260ae64b7b8968ee9b4e4fd983/users/3e671ad0b7bf4da0ab4bc729b8f6db2b/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:54.660242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b258f-bf7a-4f33-8362-30d9f60d623a admin admin] RBAC: Authorizing `identity:create_grant(project_id=43bd55a6e88b4038a0772445761e7135, user_id=fd3c2ae863ad4c54aeb2c652d6f7e90f, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:54.662571 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69372cc5-7b00-42d9-aa99-54e18831aec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f663d2260ae64b7b8968ee9b4e4fd983, user_id=3e671ad0b7bf4da0ab4bc729b8f6db2b, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:54.665334 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69372cc5-7b00-42d9-aa99-54e18831aec4 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:54.668407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b258f-bf7a-4f33-8362-30d9f60d623a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:54.773214 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 807/1631] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:54 2022] PUT /identity/v3/projects/43bd55a6e88b4038a0772445761e7135/users/fd3c2ae863ad4c54aeb2c652d6f7e90f/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:54.777179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b258f-bf7a-4f33-8362-30d9f60d623a admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:54.818844 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 825/1632] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:54 2022] PUT /identity/v3/projects/f663d2260ae64b7b8968ee9b4e4fd983/users/3e671ad0b7bf4da0ab4bc729b8f6db2b/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:54.822479 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69372cc5-7b00-42d9-aa99-54e18831aec4 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:54.937672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b258f-bf7a-4f33-8362-30d9f60d623a admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:54.941926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69372cc5-7b00-42d9-aa99-54e18831aec4 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:54.955487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bebe9-1e37-4f1a-a7bc-4e083d38c526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:54.956132 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bebe9-1e37-4f1a-a7bc-4e083d38c526 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:54.956417 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bebe9-1e37-4f1a-a7bc-4e083d38c526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:54.956696 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bebe9-1e37-4f1a-a7bc-4e083d38c526 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:54.957379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3bebe9-1e37-4f1a-a7bc-4e083d38c526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:54.965232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3bebe9-1e37-4f1a-a7bc-4e083d38c526 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:54.967505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac74056-ebbd-48d4-8acb-e33b6f2aaa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:54.968273 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac74056-ebbd-48d4-8acb-e33b6f2aaa75 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:54.968560 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac74056-ebbd-48d4-8acb-e33b6f2aaa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:54.968837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac74056-ebbd-48d4-8acb-e33b6f2aaa75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:54.969692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac74056-ebbd-48d4-8acb-e33b6f2aaa75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:54.971519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac74056-ebbd-48d4-8acb-e33b6f2aaa75 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:54.993558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 826/1633] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:54.993558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c0211-5dc8-40aa-b6ab-79121d4982e9 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:54.993558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c0211-5dc8-40aa-b6ab-79121d4982e9 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:54.993558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c0211-5dc8-40aa-b6ab-79121d4982e9 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:54.994387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 827/1634] 10.222.0.59 () {54 vars in 912 bytes} [Wed Aug 31 19:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:24:54.995235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c0211-5dc8-40aa-b6ab-79121d4982e9 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:54.995558 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 808/1635] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:24:55.000190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac74056-ebbd-48d4-8acb-e33b6f2aaa75 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:55.013635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c0211-5dc8-40aa-b6ab-79121d4982e9 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:55.016197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac74056-ebbd-48d4-8acb-e33b6f2aaa75 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:55.020370 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012a9e2-9550-4b15-acd0-5cfec2170873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:55.021518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012a9e2-9550-4b15-acd0-5cfec2170873 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:55.021811 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012a9e2-9550-4b15-acd0-5cfec2170873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:55.022090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012a9e2-9550-4b15-acd0-5cfec2170873 admin admin] PATH_INFO: `/v3/projects/f663d2260ae64b7b8968ee9b4e4fd983/users/3e671ad0b7bf4da0ab4bc729b8f6db2b/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:55.022629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434235f5-9ee6-484f-b8e9-a943536485d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:55.023536 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434235f5-9ee6-484f-b8e9-a943536485d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:55.023884 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434235f5-9ee6-484f-b8e9-a943536485d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:55.024169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434235f5-9ee6-484f-b8e9-a943536485d9 admin admin] PATH_INFO: `/v3/projects/43bd55a6e88b4038a0772445761e7135/users/fd3c2ae863ad4c54aeb2c652d6f7e90f/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:55.026858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012a9e2-9550-4b15-acd0-5cfec2170873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f663d2260ae64b7b8968ee9b4e4fd983, user_id=3e671ad0b7bf4da0ab4bc729b8f6db2b, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:55.028441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434235f5-9ee6-484f-b8e9-a943536485d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43bd55a6e88b4038a0772445761e7135, user_id=fd3c2ae863ad4c54aeb2c652d6f7e90f, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:55.030666 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434235f5-9ee6-484f-b8e9-a943536485d9 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:55.031721 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012a9e2-9550-4b15-acd0-5cfec2170873 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:55.048575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 828/1636] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:54 2022] PUT /identity/v3/projects/f663d2260ae64b7b8968ee9b4e4fd983/users/3e671ad0b7bf4da0ab4bc729b8f6db2b/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:55.050123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ecc9f-5cb7-4b72-b888-3908c9405166 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:55.051411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ecc9f-5cb7-4b72-b888-3908c9405166 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:55.051723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ecc9f-5cb7-4b72-b888-3908c9405166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:55.070582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 809/1637] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:54 2022] PUT /identity/v3/projects/43bd55a6e88b4038a0772445761e7135/users/fd3c2ae863ad4c54aeb2c652d6f7e90f/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:55.071882 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7190eaf-535f-48c7-ad0c-69c87519907e None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:55.072027 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7190eaf-535f-48c7-ad0c-69c87519907e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:55.072360 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7190eaf-535f-48c7-ad0c-69c87519907e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:55.084802 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d31ecc9f-5cb7-4b72-b888-3908c9405166 None None] MFA Rules not processed for user `d461422ce98442bcbe6b254a8dad5e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:55.086071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d31ecc9f-5cb7-4b72-b888-3908c9405166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:55.092456 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:55.109659 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-d7190eaf-535f-48c7-ad0c-69c87519907e None None] MFA Rules not processed for user `3e671ad0b7bf4da0ab4bc729b8f6db2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:55.110938 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d7190eaf-535f-48c7-ad0c-69c87519907e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:55.213273 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 810/1638] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:55 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:55.213273 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed4eb9-5da3-47ae-89ed-77e29a203ad1 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:55.213273 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed4eb9-5da3-47ae-89ed-77e29a203ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:55.213273 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed4eb9-5da3-47ae-89ed-77e29a203ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:55.222252 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 829/1639] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 31 19:24:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:55.225120 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ecc9f-5cb7-4b72-b888-3908c9405166 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:55.254902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-13ed4eb9-5da3-47ae-89ed-77e29a203ad1 None None] MFA Rules not processed for user `fd3c2ae863ad4c54aeb2c652d6f7e90f`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:55.257609 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-13ed4eb9-5da3-47ae-89ed-77e29a203ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:55.345142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:55.361245 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ecc9f-5cb7-4b72-b888-3908c9405166 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:55.364222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 811/1640] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:55.369734 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cee9e9-c34d-4ee5-8588-a37d1ff177cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:55.371521 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cee9e9-c34d-4ee5-8588-a37d1ff177cd service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:55.372074 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cee9e9-c34d-4ee5-8588-a37d1ff177cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:55.372614 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cee9e9-c34d-4ee5-8588-a37d1ff177cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:55.386179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cee9e9-c34d-4ee5-8588-a37d1ff177cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:55.388244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cee9e9-c34d-4ee5-8588-a37d1ff177cd service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:55.400535 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 830/1641] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:55 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:55.478163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed4eb9-5da3-47ae-89ed-77e29a203ad1 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:55.490838 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed4eb9-5da3-47ae-89ed-77e29a203ad1 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:55.502108 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7670727-907a-4832-9a6f-b7ee8b4b832c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:55.502838 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7670727-907a-4832-9a6f-b7ee8b4b832c service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:55.504492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7670727-907a-4832-9a6f-b7ee8b4b832c service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:55.504899 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7670727-907a-4832-9a6f-b7ee8b4b832c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:55.531580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7670727-907a-4832-9a6f-b7ee8b4b832c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:55.534145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7670727-907a-4832-9a6f-b7ee8b4b832c service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:55.571046 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 812/1642] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:55.643227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cee9e9-c34d-4ee5-8588-a37d1ff177cd service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:55.666140 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cee9e9-c34d-4ee5-8588-a37d1ff177cd service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:55.679694 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0ae29-5731-4790-b516-bbf7edbaf010 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:55.681702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0ae29-5731-4790-b516-bbf7edbaf010 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:55.683467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0ae29-5731-4790-b516-bbf7edbaf010 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:55.683870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0ae29-5731-4790-b516-bbf7edbaf010 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:55.707018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0ae29-5731-4790-b516-bbf7edbaf010 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:55.711816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0ae29-5731-4790-b516-bbf7edbaf010 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:55.730333 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 831/1643] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:56.422796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7670727-907a-4832-9a6f-b7ee8b4b832c service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:56.422796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7670727-907a-4832-9a6f-b7ee8b4b832c service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:56.422796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ca5d1-d4d4-4d8d-92c7-2860aaf41843 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:56.422796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ca5d1-d4d4-4d8d-92c7-2860aaf41843 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:56.428810 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ca5d1-d4d4-4d8d-92c7-2860aaf41843 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:56.428810 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ca5d1-d4d4-4d8d-92c7-2860aaf41843 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:56.491940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ca5d1-d4d4-4d8d-92c7-2860aaf41843 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:56.493181 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ca5d1-d4d4-4d8d-92c7-2860aaf41843 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:56.503472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 813/1644] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:56.751983 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0ae29-5731-4790-b516-bbf7edbaf010 service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:56.875773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0ae29-5731-4790-b516-bbf7edbaf010 service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:56.888666 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd187957-79e2-4416-b87b-7e255716c995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:56.890110 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd187957-79e2-4416-b87b-7e255716c995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:56.890391 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd187957-79e2-4416-b87b-7e255716c995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:56.890665 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd187957-79e2-4416-b87b-7e255716c995 admin admin] PATH_INFO: `/v3/users/ef8c73aa3c7e4adca83cd151fd88fae1` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:56.897334 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd187957-79e2-4416-b87b-7e255716c995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef8c73aa3c7e4adca83cd151fd88fae1)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:56.898387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: WARNING py.warnings [None req-fd187957-79e2-4416-b87b-7e255716c995 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:24:56.898387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: warnings.warn(msg) Aug 31 19:24:56.898387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: Aug 31 19:24:56.910026 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd187957-79e2-4416-b87b-7e255716c995 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:57.191192 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-fd187957-79e2-4416-b87b-7e255716c995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef8c73aa3c7e4adca83cd151fd88fae1'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:57.244837 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-fd187957-79e2-4416-b87b-7e255716c995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef8c73aa3c7e4adca83cd151fd88fae1'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:57.353373 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-fd187957-79e2-4416-b87b-7e255716c995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef8c73aa3c7e4adca83cd151fd88fae1'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:57.367322 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-fd187957-79e2-4416-b87b-7e255716c995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef8c73aa3c7e4adca83cd151fd88fae1'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:57.368130 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-fd187957-79e2-4416-b87b-7e255716c995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef8c73aa3c7e4adca83cd151fd88fae1'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:57.374846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 832/1645] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:24:56 2022] DELETE /identity/v3/users/ef8c73aa3c7e4adca83cd151fd88fae1 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:24:57.380074 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a9e8c-2862-435c-8730-181430cd07be None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:57.380074 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a9e8c-2862-435c-8730-181430cd07be None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:57.380320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a9e8c-2862-435c-8730-181430cd07be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:57.387474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:57.402589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-e48a9e8c-2862-435c-8730-181430cd07be None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:57.404042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e48a9e8c-2862-435c-8730-181430cd07be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:57.527811 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 814/1646] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:57.548920 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd187957-79e2-4416-b87b-7e255716c995 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:57.552314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-fd187957-79e2-4416-b87b-7e255716c995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:57.710304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd187957-79e2-4416-b87b-7e255716c995 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:57.715109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd3946-b622-4e61-8a27-8b143b632c0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:57.716222 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd3946-b622-4e61-8a27-8b143b632c0f service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:57.717810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd3946-b622-4e61-8a27-8b143b632c0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:57.718486 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd3946-b622-4e61-8a27-8b143b632c0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:57.802539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:57.809178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd3946-b622-4e61-8a27-8b143b632c0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:57.820220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd3946-b622-4e61-8a27-8b143b632c0f service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:57.838695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 833/1647] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:59.226237 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a9e8c-2862-435c-8730-181430cd07be None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:59.228400 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e48a9e8c-2862-435c-8730-181430cd07be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:59.261708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a9e8c-2862-435c-8730-181430cd07be None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:59.269708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:59.271165 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:59.271737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:59.272160 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8 admin admin] PATH_INFO: `/v3/projects/a5258453bce34fb680a34963e4bbe733` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:59.273546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5258453bce34fb680a34963e4bbe733)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:59.275384 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: WARNING py.warnings [None req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:24:59.275384 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: warnings.warn(msg) Aug 31 19:24:59.275384 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: Aug 31 19:24:59.278488 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:59.491460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5258453bce34fb680a34963e4bbe733', 'request_id': 'req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:59.502566 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5258453bce34fb680a34963e4bbe733', 'request_id': 'req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:59.518861 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5258453bce34fb680a34963e4bbe733', 'request_id': 'req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:59.647886 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8 admin admin] The token cache is being invalidate because project a5258453bce34fb680a34963e4bbe733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:24:59.648200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:59.650294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:59.656851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 815/1648] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:24:59 2022] DELETE /identity/v3/projects/a5258453bce34fb680a34963e4bbe733 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:59.660457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd3946-b622-4e61-8a27-8b143b632c0f service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:59.677495 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd3946-b622-4e61-8a27-8b143b632c0f service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:59.802391 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a339b-2d3b-4777-bac6-5392a70c3799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:59.803880 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a339b-2d3b-4777-bac6-5392a70c3799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:59.804175 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a339b-2d3b-4777-bac6-5392a70c3799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:59.804512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a339b-2d3b-4777-bac6-5392a70c3799 admin admin] PATH_INFO: `/v3/users/3e671ad0b7bf4da0ab4bc729b8f6db2b` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:59.806003 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a339b-2d3b-4777-bac6-5392a70c3799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e671ad0b7bf4da0ab4bc729b8f6db2b)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:59.808306 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a339b-2d3b-4777-bac6-5392a70c3799 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:00.035300 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d77a339b-2d3b-4777-bac6-5392a70c3799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e671ad0b7bf4da0ab4bc729b8f6db2b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:00.094238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d77a339b-2d3b-4777-bac6-5392a70c3799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e671ad0b7bf4da0ab4bc729b8f6db2b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:00.198593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d77a339b-2d3b-4777-bac6-5392a70c3799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e671ad0b7bf4da0ab4bc729b8f6db2b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:00.214508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d77a339b-2d3b-4777-bac6-5392a70c3799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e671ad0b7bf4da0ab4bc729b8f6db2b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:00.218454 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d77a339b-2d3b-4777-bac6-5392a70c3799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e671ad0b7bf4da0ab4bc729b8f6db2b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:00.240836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 834/1649] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:24:59 2022] DELETE /identity/v3/users/3e671ad0b7bf4da0ab4bc729b8f6db2b => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:00.512870 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:00.514846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:00.533354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68c452-48df-44d5-b107-4c3c7e4fb4d8 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:00.667304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a1e23-a420-40d4-9a4c-c7b826502951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:00.668103 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a1e23-a420-40d4-9a4c-c7b826502951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:00.668517 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a1e23-a420-40d4-9a4c-c7b826502951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:00.668867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a1e23-a420-40d4-9a4c-c7b826502951 admin admin] PATH_INFO: `/v3/users/793b30c8c0df4e789e745fe4a1a10eea` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:00.670289 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a1e23-a420-40d4-9a4c-c7b826502951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=793b30c8c0df4e789e745fe4a1a10eea)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:00.672931 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a1e23-a420-40d4-9a4c-c7b826502951 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:00.775867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:00.901444 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-5d2a1e23-a420-40d4-9a4c-c7b826502951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '793b30c8c0df4e789e745fe4a1a10eea'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:01.091861 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-5d2a1e23-a420-40d4-9a4c-c7b826502951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '793b30c8c0df4e789e745fe4a1a10eea'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:01.093244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-5d2a1e23-a420-40d4-9a4c-c7b826502951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '793b30c8c0df4e789e745fe4a1a10eea'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:01.114030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-5d2a1e23-a420-40d4-9a4c-c7b826502951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '793b30c8c0df4e789e745fe4a1a10eea'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:01.117813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-5d2a1e23-a420-40d4-9a4c-c7b826502951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '793b30c8c0df4e789e745fe4a1a10eea'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:01.161496 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 816/1650] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:00 2022] DELETE /identity/v3/users/793b30c8c0df4e789e745fe4a1a10eea => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:01.164251 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14949694-f64f-413d-874c-01bc4a03be94 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:01.165449 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14949694-f64f-413d-874c-01bc4a03be94 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:01.166105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14949694-f64f-413d-874c-01bc4a03be94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:01.204724 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-14949694-f64f-413d-874c-01bc4a03be94 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:01.210121 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-14949694-f64f-413d-874c-01bc4a03be94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:01.418385 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 835/1651] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:01.456753 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a1e23-a420-40d4-9a4c-c7b826502951 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:01.459004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-5d2a1e23-a420-40d4-9a4c-c7b826502951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:01.573440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a1e23-a420-40d4-9a4c-c7b826502951 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:01.581200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c742c0-bfa4-4a5f-a743-6fe642eb5fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:01.587130 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c742c0-bfa4-4a5f-a743-6fe642eb5fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:01.587512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c742c0-bfa4-4a5f-a743-6fe642eb5fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:01.587967 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c742c0-bfa4-4a5f-a743-6fe642eb5fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:01.608687 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c742c0-bfa4-4a5f-a743-6fe642eb5fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:01.614087 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c742c0-bfa4-4a5f-a743-6fe642eb5fb0 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:01.659143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 817/1652] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:02.934603 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14949694-f64f-413d-874c-01bc4a03be94 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:02.937993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-14949694-f64f-413d-874c-01bc4a03be94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:02.973433 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14949694-f64f-413d-874c-01bc4a03be94 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:02.996135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb3a66-16a4-468c-9fc5-485d6f8cea56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:02.996868 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb3a66-16a4-468c-9fc5-485d6f8cea56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:02.997154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb3a66-16a4-468c-9fc5-485d6f8cea56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:02.998785 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb3a66-16a4-468c-9fc5-485d6f8cea56 admin admin] PATH_INFO: `/v3/projects/f663d2260ae64b7b8968ee9b4e4fd983` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:03.002172 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb3a66-16a4-468c-9fc5-485d6f8cea56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f663d2260ae64b7b8968ee9b4e4fd983)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:03.010450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb3a66-16a4-468c-9fc5-485d6f8cea56 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:03.038794 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:03.153916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c742c0-bfa4-4a5f-a743-6fe642eb5fb0 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:03.156365 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-62c742c0-bfa4-4a5f-a743-6fe642eb5fb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:03.229576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c742c0-bfa4-4a5f-a743-6fe642eb5fb0 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:03.234204 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f2bb3a66-16a4-468c-9fc5-485d6f8cea56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f663d2260ae64b7b8968ee9b4e4fd983', 'request_id': 'req-f2bb3a66-16a4-468c-9fc5-485d6f8cea56'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:03.247269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f2bb3a66-16a4-468c-9fc5-485d6f8cea56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f663d2260ae64b7b8968ee9b4e4fd983', 'request_id': 'req-f2bb3a66-16a4-468c-9fc5-485d6f8cea56'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:03.348900 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f2bb3a66-16a4-468c-9fc5-485d6f8cea56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f663d2260ae64b7b8968ee9b4e4fd983', 'request_id': 'req-f2bb3a66-16a4-468c-9fc5-485d6f8cea56'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:03.371753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f2bb3a66-16a4-468c-9fc5-485d6f8cea56 admin admin] The token cache is being invalidate because project f663d2260ae64b7b8968ee9b4e4fd983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:03.372284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f2bb3a66-16a4-468c-9fc5-485d6f8cea56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:03.373621 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f2bb3a66-16a4-468c-9fc5-485d6f8cea56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:03.375978 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 836/1653] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:02 2022] DELETE /identity/v3/projects/f663d2260ae64b7b8968ee9b4e4fd983 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:03.382069 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a3ea0-652f-486a-b53c-4cba1d67791f None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:03.384608 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a3ea0-652f-486a-b53c-4cba1d67791f None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:03.385474 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a3ea0-652f-486a-b53c-4cba1d67791f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:03.416947 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-882a3ea0-652f-486a-b53c-4cba1d67791f None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:03.418886 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-882a3ea0-652f-486a-b53c-4cba1d67791f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:03.438510 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26af35d-9132-4e26-81e0-6fc2cf907188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:03.439205 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26af35d-9132-4e26-81e0-6fc2cf907188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:03.439640 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26af35d-9132-4e26-81e0-6fc2cf907188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:03.440080 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26af35d-9132-4e26-81e0-6fc2cf907188 admin admin] PATH_INFO: `/v3/projects/7239e9b25901478398e787e38e51725f` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:03.441753 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26af35d-9132-4e26-81e0-6fc2cf907188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7239e9b25901478398e787e38e51725f)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:03.445547 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26af35d-9132-4e26-81e0-6fc2cf907188 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:03.546386 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 837/1654] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:03.554309 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accb061-6575-468a-b756-ec1f448bd284 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:03.554928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accb061-6575-468a-b756-ec1f448bd284 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:03.555351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accb061-6575-468a-b756-ec1f448bd284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:03.629256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-3accb061-6575-468a-b756-ec1f448bd284 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:03.630753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3accb061-6575-468a-b756-ec1f448bd284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:03.635073 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d26af35d-9132-4e26-81e0-6fc2cf907188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7239e9b25901478398e787e38e51725f', 'request_id': 'req-d26af35d-9132-4e26-81e0-6fc2cf907188'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:03.637000 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:03.661123 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d26af35d-9132-4e26-81e0-6fc2cf907188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7239e9b25901478398e787e38e51725f', 'request_id': 'req-d26af35d-9132-4e26-81e0-6fc2cf907188'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:03.678030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d26af35d-9132-4e26-81e0-6fc2cf907188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7239e9b25901478398e787e38e51725f', 'request_id': 'req-d26af35d-9132-4e26-81e0-6fc2cf907188'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:03.778108 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d26af35d-9132-4e26-81e0-6fc2cf907188 admin admin] The token cache is being invalidate because project 7239e9b25901478398e787e38e51725f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:03.778478 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d26af35d-9132-4e26-81e0-6fc2cf907188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:03.782544 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d26af35d-9132-4e26-81e0-6fc2cf907188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:03.795757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 818/1655] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:03 2022] DELETE /identity/v3/projects/7239e9b25901478398e787e38e51725f => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:25:03.797494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7a8f8-82e4-4fc2-8291-30d0a3d16104 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:03.797973 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7a8f8-82e4-4fc2-8291-30d0a3d16104 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:03.798270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7a8f8-82e4-4fc2-8291-30d0a3d16104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:03.803259 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 838/1656] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:25:03.807917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accb061-6575-468a-b756-ec1f448bd284 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:03.840531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accb061-6575-468a-b756-ec1f448bd284 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:03.857401 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15231800-8cc3-430f-b8e4-28a7dff024e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:03.858491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15231800-8cc3-430f-b8e4-28a7dff024e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:03.859028 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15231800-8cc3-430f-b8e4-28a7dff024e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:03.859538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15231800-8cc3-430f-b8e4-28a7dff024e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:03.860003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-4bc7a8f8-82e4-4fc2-8291-30d0a3d16104 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:03.861226 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15231800-8cc3-430f-b8e4-28a7dff024e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:25:03.861343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4bc7a8f8-82e4-4fc2-8291-30d0a3d16104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:03.862242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15231800-8cc3-430f-b8e4-28a7dff024e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:03.866483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15231800-8cc3-430f-b8e4-28a7dff024e3 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:03.890542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 819/1657] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:03.898091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3c4e9-7537-43cc-a395-9a42c06e1958 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:03.898518 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3c4e9-7537-43cc-a395-9a42c06e1958 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:03.899993 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3c4e9-7537-43cc-a395-9a42c06e1958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:03.908690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 839/1658] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:25:03.917633 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15231800-8cc3-430f-b8e4-28a7dff024e3 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:03.951481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15231800-8cc3-430f-b8e4-28a7dff024e3 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:03.972049 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-36d3c4e9-7537-43cc-a395-9a42c06e1958 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:03.972161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f139ca-a45f-4b87-8f7f-6b07decb9ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:03.973506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f139ca-a45f-4b87-8f7f-6b07decb9ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:03.974426 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f139ca-a45f-4b87-8f7f-6b07decb9ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:03.975606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f139ca-a45f-4b87-8f7f-6b07decb9ab0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:03.975924 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-36d3c4e9-7537-43cc-a395-9a42c06e1958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:03.980386 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f139ca-a45f-4b87-8f7f-6b07decb9ab0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:03.988493 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f139ca-a45f-4b87-8f7f-6b07decb9ab0 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:04.009490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 820/1659] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:04.023634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3c4e9-7537-43cc-a395-9a42c06e1958 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:04.038981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3c4e9-7537-43cc-a395-9a42c06e1958 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:04.050488 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ec704-3191-4519-9813-4c23caf55331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:04.051281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ec704-3191-4519-9813-4c23caf55331 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:04.051674 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ec704-3191-4519-9813-4c23caf55331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:04.052054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ec704-3191-4519-9813-4c23caf55331 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:04.053053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ec704-3191-4519-9813-4c23caf55331 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:25:04.053471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ec704-3191-4519-9813-4c23caf55331 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:04.055472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ec704-3191-4519-9813-4c23caf55331 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:04.071281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 821/1660] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:25:04.075251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ec704-3191-4519-9813-4c23caf55331 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:04.103010 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ec704-3191-4519-9813-4c23caf55331 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:04.127447 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 840/1661] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:03 2022] POST /identity/v3/projects => generated 365 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:25:04.128671 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f139ca-a45f-4b87-8f7f-6b07decb9ab0 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:04.130295 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b224078-2e9e-40bc-8f2c-560bf53364c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:04.134718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b224078-2e9e-40bc-8f2c-560bf53364c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:04.136998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b224078-2e9e-40bc-8f2c-560bf53364c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:04.137395 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b224078-2e9e-40bc-8f2c-560bf53364c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:04.140694 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b224078-2e9e-40bc-8f2c-560bf53364c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:04.148867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b224078-2e9e-40bc-8f2c-560bf53364c9 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:04.290950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 822/1662] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:04 2022] POST /identity/v3/projects => generated 379 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:25:04.294948 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b224078-2e9e-40bc-8f2c-560bf53364c9 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:04.319388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f139ca-a45f-4b87-8f7f-6b07decb9ab0 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:04.390199 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:04.441849 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93dae7a-f9ed-49b7-ba81-d79a422b0bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:04.442629 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93dae7a-f9ed-49b7-ba81-d79a422b0bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:04.442998 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93dae7a-f9ed-49b7-ba81-d79a422b0bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:04.443265 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93dae7a-f9ed-49b7-ba81-d79a422b0bf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:04.444245 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93dae7a-f9ed-49b7-ba81-d79a422b0bf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:04.446229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93dae7a-f9ed-49b7-ba81-d79a422b0bf3 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:04.483792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b224078-2e9e-40bc-8f2c-560bf53364c9 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:04.494320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf28dd5-e56a-4f05-ba57-6307574d723a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:04.495072 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf28dd5-e56a-4f05-ba57-6307574d723a admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:04.496079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf28dd5-e56a-4f05-ba57-6307574d723a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:04.496079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf28dd5-e56a-4f05-ba57-6307574d723a admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:04.496890 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf28dd5-e56a-4f05-ba57-6307574d723a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:04.510181 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf28dd5-e56a-4f05-ba57-6307574d723a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:04.549281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 841/1663] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:04 2022] POST /identity/v3/users => generated 283 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:04.553628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93dae7a-f9ed-49b7-ba81-d79a422b0bf3 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:04.577428 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93dae7a-f9ed-49b7-ba81-d79a422b0bf3 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:04.590569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f18c4-88ff-4e79-9705-bb29f86f185d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:04.591908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f18c4-88ff-4e79-9705-bb29f86f185d admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:04.593415 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f18c4-88ff-4e79-9705-bb29f86f185d admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:04.594254 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f18c4-88ff-4e79-9705-bb29f86f185d admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:04.595933 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f18c4-88ff-4e79-9705-bb29f86f185d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:04.598266 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f18c4-88ff-4e79-9705-bb29f86f185d admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:04.622315 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 842/1664] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:04.623645 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 823/1665] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:04 2022] POST /identity/v3/users => generated 290 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:04.628513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f18c4-88ff-4e79-9705-bb29f86f185d admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:04.628912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf28dd5-e56a-4f05-ba57-6307574d723a admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:04.651246 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f18c4-88ff-4e79-9705-bb29f86f185d admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:04.653719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf28dd5-e56a-4f05-ba57-6307574d723a admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:04.664003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa630769-fb46-4586-94ee-8040cbce2b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:04.664756 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630769-fb46-4586-94ee-8040cbce2b36 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:04.665145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630769-fb46-4586-94ee-8040cbce2b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:04.665623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630769-fb46-4586-94ee-8040cbce2b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:04.666618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa630769-fb46-4586-94ee-8040cbce2b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:04.669091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa630769-fb46-4586-94ee-8040cbce2b36 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:04.683773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c247a01-d422-43d9-9cd3-264e8553c5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:04.687339 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c247a01-d422-43d9-9cd3-264e8553c5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:04.687824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c247a01-d422-43d9-9cd3-264e8553c5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:04.688272 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c247a01-d422-43d9-9cd3-264e8553c5f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:04.689145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c247a01-d422-43d9-9cd3-264e8553c5f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:04.691266 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 824/1666] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:04.692380 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa630769-fb46-4586-94ee-8040cbce2b36 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:04.694520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c247a01-d422-43d9-9cd3-264e8553c5f0 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:04.722651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa630769-fb46-4586-94ee-8040cbce2b36 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:04.723743 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 843/1667] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:04.726714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c247a01-d422-43d9-9cd3-264e8553c5f0 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:04.744429 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf366d7-0e7f-4d09-9971-2d6e5b75a02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:04.746282 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf366d7-0e7f-4d09-9971-2d6e5b75a02a admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:04.746578 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf366d7-0e7f-4d09-9971-2d6e5b75a02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:04.748620 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf366d7-0e7f-4d09-9971-2d6e5b75a02a admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:04.750292 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf366d7-0e7f-4d09-9971-2d6e5b75a02a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:04.755335 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf366d7-0e7f-4d09-9971-2d6e5b75a02a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:04.759432 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c247a01-d422-43d9-9cd3-264e8553c5f0 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:04.772643 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17110853-f549-4a1d-909a-daf12bb36725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:04.773419 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17110853-f549-4a1d-909a-daf12bb36725 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:04.774089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17110853-f549-4a1d-909a-daf12bb36725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:04.779388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17110853-f549-4a1d-909a-daf12bb36725 admin admin] PATH_INFO: `/v3/projects/293acde59f5a418b8da3e49a9ee76893/users/a5126c8e7b8546eca203ed30a85a89fb/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:04.810362 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17110853-f549-4a1d-909a-daf12bb36725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=293acde59f5a418b8da3e49a9ee76893, user_id=a5126c8e7b8546eca203ed30a85a89fb, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:04.814530 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17110853-f549-4a1d-909a-daf12bb36725 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:04.821628 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 825/1668] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:04.822581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf366d7-0e7f-4d09-9971-2d6e5b75a02a admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:04.856426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf366d7-0e7f-4d09-9971-2d6e5b75a02a admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:04.877998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1c143-d934-4fb9-89af-f4709897be5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:04.878955 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1c143-d934-4fb9-89af-f4709897be5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:04.886790 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1c143-d934-4fb9-89af-f4709897be5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:04.886790 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1c143-d934-4fb9-89af-f4709897be5d admin admin] PATH_INFO: `/v3/projects/1eb0d93260b44c6ea812bcc790e24dab/users/324faeccb7fa41c093bfae329fd960d0/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:04.917337 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 844/1669] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:04 2022] PUT /identity/v3/projects/293acde59f5a418b8da3e49a9ee76893/users/a5126c8e7b8546eca203ed30a85a89fb/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:04.926316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1c143-d934-4fb9-89af-f4709897be5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eb0d93260b44c6ea812bcc790e24dab, user_id=324faeccb7fa41c093bfae329fd960d0, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:04.928632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a9a50-c36d-4bc8-897a-048310268771 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:04.929032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a9a50-c36d-4bc8-897a-048310268771 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:04.929416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a9a50-c36d-4bc8-897a-048310268771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:04.933706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1c143-d934-4fb9-89af-f4709897be5d admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:05.017859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-839a9a50-c36d-4bc8-897a-048310268771 None None] MFA Rules not processed for user `a5126c8e7b8546eca203ed30a85a89fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:05.022255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-839a9a50-c36d-4bc8-897a-048310268771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:05.031209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 826/1670] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:04 2022] PUT /identity/v3/projects/1eb0d93260b44c6ea812bcc790e24dab/users/324faeccb7fa41c093bfae329fd960d0/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:05.042990 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46b7ae-f622-4422-b148-fbe49f3da995 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:05.043284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46b7ae-f622-4422-b148-fbe49f3da995 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:05.043568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46b7ae-f622-4422-b148-fbe49f3da995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:05.111834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-9f46b7ae-f622-4422-b148-fbe49f3da995 None None] MFA Rules not processed for user `324faeccb7fa41c093bfae329fd960d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:05.113087 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-9f46b7ae-f622-4422-b148-fbe49f3da995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:05.187176 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 845/1671] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:25:05.192348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a9a50-c36d-4bc8-897a-048310268771 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:05.301335 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 827/1672] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:05.303911 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46b7ae-f622-4422-b148-fbe49f3da995 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:05.311966 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:DELETE=3,keystone:SELECT=91,keystone:INSERT=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:05.419069 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a9a50-c36d-4bc8-897a-048310268771 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:05.444204 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a33f4-5cba-4392-97af-68f71788f72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:05.444848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a33f4-5cba-4392-97af-68f71788f72e admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:05.445132 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a33f4-5cba-4392-97af-68f71788f72e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:05.445409 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a33f4-5cba-4392-97af-68f71788f72e admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:05.446066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a33f4-5cba-4392-97af-68f71788f72e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:05.450016 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a33f4-5cba-4392-97af-68f71788f72e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:05.459152 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46b7ae-f622-4422-b148-fbe49f3da995 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:05.476296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f09f0-de3e-448f-a479-568976596ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:05.477276 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f09f0-de3e-448f-a479-568976596ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:05.477535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f09f0-de3e-448f-a479-568976596ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:05.477815 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f09f0-de3e-448f-a479-568976596ee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:05.494278 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f09f0-de3e-448f-a479-568976596ee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:05.496550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f09f0-de3e-448f-a479-568976596ee2 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:05.642788 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 846/1673] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:05 2022] POST /identity/v3/projects => generated 363 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:05.646662 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a33f4-5cba-4392-97af-68f71788f72e admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:05.697105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 828/1674] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:05 2022] POST /identity/v3/projects => generated 381 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:05.702021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f09f0-de3e-448f-a479-568976596ee2 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:05.944476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f09f0-de3e-448f-a479-568976596ee2 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:05.964514 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a33f4-5cba-4392-97af-68f71788f72e admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:05.971668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d41b4-d964-4e87-aacc-0ee1380ddbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:05.972318 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d41b4-d964-4e87-aacc-0ee1380ddbf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:05.972786 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d41b4-d964-4e87-aacc-0ee1380ddbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:05.974029 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d41b4-d964-4e87-aacc-0ee1380ddbf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:05.975472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d41b4-d964-4e87-aacc-0ee1380ddbf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:05.979652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d41b4-d964-4e87-aacc-0ee1380ddbf8 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:05.984057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e53ad2-0743-4211-82ae-d89aed36026a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:05.984828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e53ad2-0743-4211-82ae-d89aed36026a admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:05.985216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e53ad2-0743-4211-82ae-d89aed36026a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:05.987096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e53ad2-0743-4211-82ae-d89aed36026a admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:05.990124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e53ad2-0743-4211-82ae-d89aed36026a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:05.993773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e53ad2-0743-4211-82ae-d89aed36026a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:06.045553 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:06.111008 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 829/1675] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:05 2022] POST /identity/v3/users => generated 297 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:06.114960 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d41b4-d964-4e87-aacc-0ee1380ddbf8 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:06.145458 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d41b4-d964-4e87-aacc-0ee1380ddbf8 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:06.165691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8bbf2-0980-4bc9-9af2-2e73627d9236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:06.166527 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8bbf2-0980-4bc9-9af2-2e73627d9236 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:06.167436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8bbf2-0980-4bc9-9af2-2e73627d9236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:06.173250 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8bbf2-0980-4bc9-9af2-2e73627d9236 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:06.174333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8bbf2-0980-4bc9-9af2-2e73627d9236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:06.175687 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8bbf2-0980-4bc9-9af2-2e73627d9236 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:06.197711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 830/1676] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:25:06.201555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 847/1677] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:05 2022] POST /identity/v3/users => generated 288 bytes in 555 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:06.203239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8bbf2-0980-4bc9-9af2-2e73627d9236 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:06.205737 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e53ad2-0743-4211-82ae-d89aed36026a admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:06.237076 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e53ad2-0743-4211-82ae-d89aed36026a admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:06.242398 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8bbf2-0980-4bc9-9af2-2e73627d9236 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:06.258147 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f0eff-739a-46f5-bf34-b1999a911fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:06.258946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf0c3f-d66f-420c-acc7-0c2a3bddbcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:06.259655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f0eff-739a-46f5-bf34-b1999a911fcb admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:06.260408 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf0c3f-d66f-420c-acc7-0c2a3bddbcc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:06.260781 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f0eff-739a-46f5-bf34-b1999a911fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:06.267381 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf0c3f-d66f-420c-acc7-0c2a3bddbcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:06.267877 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f0eff-739a-46f5-bf34-b1999a911fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:06.268464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf0c3f-d66f-420c-acc7-0c2a3bddbcc4 admin admin] PATH_INFO: `/v3/projects/cc78554a7d494875bde5f29ff8b9b634/users/6b343dd958204fce9bd43f0809bd0e9f/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:06.269325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f0eff-739a-46f5-bf34-b1999a911fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:06.272063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f0eff-739a-46f5-bf34-b1999a911fcb admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:06.302960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 848/1678] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:25:06.311381 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f0eff-739a-46f5-bf34-b1999a911fcb admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:06.332978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf0c3f-d66f-420c-acc7-0c2a3bddbcc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc78554a7d494875bde5f29ff8b9b634, user_id=6b343dd958204fce9bd43f0809bd0e9f, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:06.332978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf0c3f-d66f-420c-acc7-0c2a3bddbcc4 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:06.337618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f0eff-739a-46f5-bf34-b1999a911fcb admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:06.347220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1040fc-dec2-4d58-9d33-4d1dcbd73cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:06.348234 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1040fc-dec2-4d58-9d33-4d1dcbd73cfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:06.348766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1040fc-dec2-4d58-9d33-4d1dcbd73cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:06.349187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1040fc-dec2-4d58-9d33-4d1dcbd73cfd admin admin] PATH_INFO: `/v3/projects/de0f3169682546cbb9e326cd899a4460/users/384f8c00cb2648a8879bbeddae5c136a/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:06.394179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1040fc-dec2-4d58-9d33-4d1dcbd73cfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=de0f3169682546cbb9e326cd899a4460, user_id=384f8c00cb2648a8879bbeddae5c136a, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:06.407182 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1040fc-dec2-4d58-9d33-4d1dcbd73cfd admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:06.413720 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 831/1679] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:06 2022] PUT /identity/v3/projects/cc78554a7d494875bde5f29ff8b9b634/users/6b343dd958204fce9bd43f0809bd0e9f/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:06.417486 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf0c3f-d66f-420c-acc7-0c2a3bddbcc4 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:06.498930 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 849/1680] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:06 2022] PUT /identity/v3/projects/de0f3169682546cbb9e326cd899a4460/users/384f8c00cb2648a8879bbeddae5c136a/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:06.504013 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1040fc-dec2-4d58-9d33-4d1dcbd73cfd admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:06.640189 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf0c3f-d66f-420c-acc7-0c2a3bddbcc4 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:06.645936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:06.694358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1040fc-dec2-4d58-9d33-4d1dcbd73cfd admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:06.711693 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427004c9-bd91-4d6b-937f-016cbdfcf47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:06.712585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427004c9-bd91-4d6b-937f-016cbdfcf47b admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:06.713669 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427004c9-bd91-4d6b-937f-016cbdfcf47b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:06.722623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427004c9-bd91-4d6b-937f-016cbdfcf47b admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:06.723881 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427004c9-bd91-4d6b-937f-016cbdfcf47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:06.729124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427004c9-bd91-4d6b-937f-016cbdfcf47b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:06.761912 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 850/1681] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:06.762899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427004c9-bd91-4d6b-937f-016cbdfcf47b admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:06.796876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427004c9-bd91-4d6b-937f-016cbdfcf47b admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:06.815325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac43367-08f4-486e-bebd-cc87cda802d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:06.815325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac43367-08f4-486e-bebd-cc87cda802d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:06.815325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac43367-08f4-486e-bebd-cc87cda802d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:06.815325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac43367-08f4-486e-bebd-cc87cda802d8 admin admin] PATH_INFO: `/v3/projects/de0f3169682546cbb9e326cd899a4460/users/384f8c00cb2648a8879bbeddae5c136a/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:06.820994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac43367-08f4-486e-bebd-cc87cda802d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de0f3169682546cbb9e326cd899a4460, user_id=384f8c00cb2648a8879bbeddae5c136a, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:06.850787 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac43367-08f4-486e-bebd-cc87cda802d8 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:06.852873 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78683498-3076-4630-a848-204cf4cdd927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:06.854087 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78683498-3076-4630-a848-204cf4cdd927 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:06.855468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78683498-3076-4630-a848-204cf4cdd927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:06.855796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78683498-3076-4630-a848-204cf4cdd927 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:06.856575 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78683498-3076-4630-a848-204cf4cdd927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:06.858955 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78683498-3076-4630-a848-204cf4cdd927 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:06.859337 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:06.898628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 851/1682] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:06 2022] PUT /identity/v3/projects/de0f3169682546cbb9e326cd899a4460/users/384f8c00cb2648a8879bbeddae5c136a/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:06.912052 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 832/1683] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 490 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:06.914031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac43367-08f4-486e-bebd-cc87cda802d8 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:06.920767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77d318-143b-4fde-86ff-00b362c9b122 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:06.921413 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77d318-143b-4fde-86ff-00b362c9b122 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:06.921806 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77d318-143b-4fde-86ff-00b362c9b122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:07.081610 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-1f77d318-143b-4fde-86ff-00b362c9b122 None None] MFA Rules not processed for user `384f8c00cb2648a8879bbeddae5c136a`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:07.083174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-1f77d318-143b-4fde-86ff-00b362c9b122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:07.195454 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac43367-08f4-486e-bebd-cc87cda802d8 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:07.214806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88571edf-7de1-436b-8cd5-be49a62f2fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:07.223361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88571edf-7de1-436b-8cd5-be49a62f2fd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:07.227919 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88571edf-7de1-436b-8cd5-be49a62f2fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:07.229465 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88571edf-7de1-436b-8cd5-be49a62f2fd2 admin admin] PATH_INFO: `/v3/projects/cc78554a7d494875bde5f29ff8b9b634/users/6b343dd958204fce9bd43f0809bd0e9f/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:07.237557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88571edf-7de1-436b-8cd5-be49a62f2fd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc78554a7d494875bde5f29ff8b9b634, user_id=6b343dd958204fce9bd43f0809bd0e9f, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:07.255160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88571edf-7de1-436b-8cd5-be49a62f2fd2 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:07.301117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 852/1684] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:06 2022] PUT /identity/v3/projects/cc78554a7d494875bde5f29ff8b9b634/users/6b343dd958204fce9bd43f0809bd0e9f/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:07.307725 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd502287-9ce9-476e-a879-bcc0e4f420f7 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:07.313509 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd502287-9ce9-476e-a879-bcc0e4f420f7 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:07.314112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd502287-9ce9-476e-a879-bcc0e4f420f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:07.389149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 833/1685] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:07.392226 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbe4fc-4f68-444f-b734-e5c82f6251e5 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:07.392819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbe4fc-4f68-444f-b734-e5c82f6251e5 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:07.393303 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbe4fc-4f68-444f-b734-e5c82f6251e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:07.427516 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-bd502287-9ce9-476e-a879-bcc0e4f420f7 None None] MFA Rules not processed for user `d441bbdb43c5403abf8480146bc15753`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:07.428778 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-bd502287-9ce9-476e-a879-bcc0e4f420f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:07.485192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-cbfbe4fc-4f68-444f-b734-e5c82f6251e5 None None] MFA Rules not processed for user `6b343dd958204fce9bd43f0809bd0e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:07.487348 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-cbfbe4fc-4f68-444f-b734-e5c82f6251e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:07.551227 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:07.634204 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 853/1686] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:07.635449 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd502287-9ce9-476e-a879-bcc0e4f420f7 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:07.642243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-bd502287-9ce9-476e-a879-bcc0e4f420f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:07.740388 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 834/1687] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:07.744944 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243e96a-335f-4369-942f-77b6aa78ce82 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:07.745906 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243e96a-335f-4369-942f-77b6aa78ce82 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:07.745906 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243e96a-335f-4369-942f-77b6aa78ce82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:07.748445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:07.800714 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-c243e96a-335f-4369-942f-77b6aa78ce82 None None] MFA Rules not processed for user `d441bbdb43c5403abf8480146bc15753`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:07.805589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c243e96a-335f-4369-942f-77b6aa78ce82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:07.835018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd502287-9ce9-476e-a879-bcc0e4f420f7 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:07.840576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 835/1688] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:07.840576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243e96a-335f-4369-942f-77b6aa78ce82 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:07.842501 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c243e96a-335f-4369-942f-77b6aa78ce82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:07.844881 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1f926-777b-4a4b-a3a2-8b58287cfbf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:07.845994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1f926-777b-4a4b-a3a2-8b58287cfbf9 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:07.846417 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1f926-777b-4a4b-a3a2-8b58287cfbf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:07.846896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1f926-777b-4a4b-a3a2-8b58287cfbf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:07.876258 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243e96a-335f-4369-942f-77b6aa78ce82 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:07.890302 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7662e7a-c6fa-4c64-9851-e91d008d857f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:07.891249 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7662e7a-c6fa-4c64-9851-e91d008d857f service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:07.891849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7662e7a-c6fa-4c64-9851-e91d008d857f service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:07.892185 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7662e7a-c6fa-4c64-9851-e91d008d857f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:07.937180 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7662e7a-c6fa-4c64-9851-e91d008d857f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:07.940599 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7662e7a-c6fa-4c64-9851-e91d008d857f service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:07.971328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 836/1689] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:07.972355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7662e7a-c6fa-4c64-9851-e91d008d857f service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:07.977616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a7662e7a-c6fa-4c64-9851-e91d008d857f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:08.073274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1f926-777b-4a4b-a3a2-8b58287cfbf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:08.074887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1f926-777b-4a4b-a3a2-8b58287cfbf9 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:08.087269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 854/1690] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:08.088592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ad734-4f42-4406-aed7-6198bba6615c None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:08.088846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ad734-4f42-4406-aed7-6198bba6615c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:08.089116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ad734-4f42-4406-aed7-6198bba6615c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:08.182257 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7662e7a-c6fa-4c64-9851-e91d008d857f service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:08.192660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d543a85-d700-4244-b01d-7b490e410546 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:08.193713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d543a85-d700-4244-b01d-7b490e410546 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:08.200078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d543a85-d700-4244-b01d-7b490e410546 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:08.205322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d543a85-d700-4244-b01d-7b490e410546 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:08.211426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d543a85-d700-4244-b01d-7b490e410546 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:08.213581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d543a85-d700-4244-b01d-7b490e410546 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:08.227903 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 837/1691] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:08.233138 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-3f9ad734-4f42-4406-aed7-6198bba6615c None None] MFA Rules not processed for user `7aaa082f2b754cf5aa941586361c1877`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:08.234484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3f9ad734-4f42-4406-aed7-6198bba6615c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:08.408510 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 855/1692] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:08.419259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac57d9-f05a-4528-b2b5-d36f7cdbad94 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:08.421088 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac57d9-f05a-4528-b2b5-d36f7cdbad94 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:08.422141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac57d9-f05a-4528-b2b5-d36f7cdbad94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:08.488691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ad734-4f42-4406-aed7-6198bba6615c None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:08.501830 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ad734-4f42-4406-aed7-6198bba6615c None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:08.517572 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6785b-534c-4220-ad27-b1dc2d8873c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:08.518204 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6785b-534c-4220-ad27-b1dc2d8873c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:08.518483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6785b-534c-4220-ad27-b1dc2d8873c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:08.518785 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6785b-534c-4220-ad27-b1dc2d8873c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:08.520287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-feac57d9-f05a-4528-b2b5-d36f7cdbad94 None None] MFA Rules not processed for user `7aaa082f2b754cf5aa941586361c1877`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:08.521533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-feac57d9-f05a-4528-b2b5-d36f7cdbad94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:08.551221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 838/1693] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:08.665622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6785b-534c-4220-ad27-b1dc2d8873c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:08.667477 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6785b-534c-4220-ad27-b1dc2d8873c9 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:08.700275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 856/1694] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:08.811157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac57d9-f05a-4528-b2b5-d36f7cdbad94 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:08.812723 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-feac57d9-f05a-4528-b2b5-d36f7cdbad94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:08.849917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac57d9-f05a-4528-b2b5-d36f7cdbad94 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:08.874661 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ba426-913c-43fa-b3c5-5496a38504d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:08.874982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ba426-913c-43fa-b3c5-5496a38504d4 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:08.875262 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ba426-913c-43fa-b3c5-5496a38504d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:08.875530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ba426-913c-43fa-b3c5-5496a38504d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:08.890971 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4ba426-913c-43fa-b3c5-5496a38504d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:08.897542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4ba426-913c-43fa-b3c5-5496a38504d4 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:08.926250 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 839/1695] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:09.831930 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1d50e-c3c0-4112-97d6-17f64803574a None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:09.832836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1d50e-c3c0-4112-97d6-17f64803574a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:09.833334 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1d50e-c3c0-4112-97d6-17f64803574a None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:09.835277 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 857/1696] 10.222.0.59 () {54 vars in 912 bytes} [Wed Aug 31 19:25:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:25:09.841570 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524f560-0aba-4902-bbff-3cbc95a64333 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:09.842744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524f560-0aba-4902-bbff-3cbc95a64333 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:09.843062 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524f560-0aba-4902-bbff-3cbc95a64333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:09.882511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1d50e-c3c0-4112-97d6-17f64803574a None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:09.897253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:09.994129 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-f524f560-0aba-4902-bbff-3cbc95a64333 None None] MFA Rules not processed for user `b7a51545146846cc8284b7044b32198a`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:09.995402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f524f560-0aba-4902-bbff-3cbc95a64333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:10.114502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:10.124330 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1d50e-c3c0-4112-97d6-17f64803574a None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:10.132098 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1d459-ce06-4c6c-adea-a1606c6f9839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:10.132785 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1d459-ce06-4c6c-adea-a1606c6f9839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:10.133839 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1d459-ce06-4c6c-adea-a1606c6f9839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:10.134119 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1d459-ce06-4c6c-adea-a1606c6f9839 admin admin] PATH_INFO: `/v3/users/a5126c8e7b8546eca203ed30a85a89fb` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:10.136126 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1d459-ce06-4c6c-adea-a1606c6f9839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5126c8e7b8546eca203ed30a85a89fb)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:10.140471 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1d459-ce06-4c6c-adea-a1606c6f9839 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:10.143478 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 840/1697] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 31 19:25:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:10.171157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524f560-0aba-4902-bbff-3cbc95a64333 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:10.172933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f524f560-0aba-4902-bbff-3cbc95a64333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:10.215538 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524f560-0aba-4902-bbff-3cbc95a64333 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:10.234457 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422051a-774f-47d2-811f-51292657866a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:10.235142 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422051a-774f-47d2-811f-51292657866a service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:10.235432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422051a-774f-47d2-811f-51292657866a service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:10.235738 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422051a-774f-47d2-811f-51292657866a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:10.268084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422051a-774f-47d2-811f-51292657866a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:10.269997 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422051a-774f-47d2-811f-51292657866a service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:10.295857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 841/1698] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:10.297393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baedce-cbaa-4838-a6da-2b34bf3a2ba8 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:10.297826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baedce-cbaa-4838-a6da-2b34bf3a2ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:10.298210 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baedce-cbaa-4838-a6da-2b34bf3a2ba8 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:10.300318 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 842/1699] 10.222.0.59 () {54 vars in 912 bytes} [Wed Aug 31 19:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:25:10.306229 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba59b5-d9d1-4fa1-a1b2-02e72be3c172 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:10.306662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba59b5-d9d1-4fa1-a1b2-02e72be3c172 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:10.307081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba59b5-d9d1-4fa1-a1b2-02e72be3c172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:10.346896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-2cba59b5-d9d1-4fa1-a1b2-02e72be3c172 None None] MFA Rules not processed for user `b7a51545146846cc8284b7044b32198a`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:10.348577 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-2cba59b5-d9d1-4fa1-a1b2-02e72be3c172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:10.364968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 843/1700] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 31 19:25:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:25:10.419992 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e0a1d459-ce06-4c6c-adea-a1606c6f9839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5126c8e7b8546eca203ed30a85a89fb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:10.454659 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba59b5-d9d1-4fa1-a1b2-02e72be3c172 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:10.457951 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-2cba59b5-d9d1-4fa1-a1b2-02e72be3c172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:10.458079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e0a1d459-ce06-4c6c-adea-a1606c6f9839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5126c8e7b8546eca203ed30a85a89fb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:10.475598 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba59b5-d9d1-4fa1-a1b2-02e72be3c172 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:10.487367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e3e05-5a5e-470c-ba5d-3e13e6d02d64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:10.488804 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e3e05-5a5e-470c-ba5d-3e13e6d02d64 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:10.489502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e3e05-5a5e-470c-ba5d-3e13e6d02d64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:10.489502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e3e05-5a5e-470c-ba5d-3e13e6d02d64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:10.518350 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e3e05-5a5e-470c-ba5d-3e13e6d02d64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:10.518350 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e3e05-5a5e-470c-ba5d-3e13e6d02d64 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:10.535745 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e0a1d459-ce06-4c6c-adea-a1606c6f9839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5126c8e7b8546eca203ed30a85a89fb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:10.541153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 844/1701] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:25:10.544294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac5a05-48f9-4ab1-9601-4220cf41704c None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:10.544584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac5a05-48f9-4ab1-9601-4220cf41704c None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:10.547886 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e0a1d459-ce06-4c6c-adea-a1606c6f9839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5126c8e7b8546eca203ed30a85a89fb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:10.548668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e0a1d459-ce06-4c6c-adea-a1606c6f9839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5126c8e7b8546eca203ed30a85a89fb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:10.548946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac5a05-48f9-4ab1-9601-4220cf41704c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:10.568653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 858/1702] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:09 2022] DELETE /identity/v3/users/a5126c8e7b8546eca203ed30a85a89fb => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:10.569804 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152ac40-29a0-4b0c-afb5-0b28e1e5ae25 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:10.573756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152ac40-29a0-4b0c-afb5-0b28e1e5ae25 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:10.573756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152ac40-29a0-4b0c-afb5-0b28e1e5ae25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:10.605515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-9152ac40-29a0-4b0c-afb5-0b28e1e5ae25 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:10.607039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9152ac40-29a0-4b0c-afb5-0b28e1e5ae25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:10.625702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-4eac5a05-48f9-4ab1-9601-4220cf41704c None None] MFA Rules not processed for user `8bc4801f1f674f55b57d49e5f09205ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:10.628803 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4eac5a05-48f9-4ab1-9601-4220cf41704c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:10.736983 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 859/1703] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:10.760402 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152ac40-29a0-4b0c-afb5-0b28e1e5ae25 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:10.762068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9152ac40-29a0-4b0c-afb5-0b28e1e5ae25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:10.808260 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 845/1704] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:10.879213 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:10.966528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152ac40-29a0-4b0c-afb5-0b28e1e5ae25 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:10.983776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aefda-0397-4c48-898c-89e4c26db6f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:10.984848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aefda-0397-4c48-898c-89e4c26db6f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:10.985307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aefda-0397-4c48-898c-89e4c26db6f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:10.985761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aefda-0397-4c48-898c-89e4c26db6f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:11.016489 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aefda-0397-4c48-898c-89e4c26db6f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:11.017453 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aefda-0397-4c48-898c-89e4c26db6f7 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:11.042375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 860/1705] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:11.355991 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac5a05-48f9-4ab1-9601-4220cf41704c None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:11.359138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4eac5a05-48f9-4ab1-9601-4220cf41704c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:11.389152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aefda-0397-4c48-898c-89e4c26db6f7 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:11.390676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-435aefda-0397-4c48-898c-89e4c26db6f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:11.550277 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac5a05-48f9-4ab1-9601-4220cf41704c None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:11.572961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2105ec-717e-432e-9988-89cc744ec62a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:11.573530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2105ec-717e-432e-9988-89cc744ec62a service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:11.574832 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2105ec-717e-432e-9988-89cc744ec62a service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:11.575161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2105ec-717e-432e-9988-89cc744ec62a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:11.577648 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aefda-0397-4c48-898c-89e4c26db6f7 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:11.623357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb48759-262c-4358-b867-a21814aa742e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:11.623357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb48759-262c-4358-b867-a21814aa742e service placement] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:11.623357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb48759-262c-4358-b867-a21814aa742e service placement] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:11.623357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb48759-262c-4358-b867-a21814aa742e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:11.626077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-0fb48759-262c-4358-b867-a21814aa742e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:11.652552 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2105ec-717e-432e-9988-89cc744ec62a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:11.655535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2105ec-717e-432e-9988-89cc744ec62a service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:11.659242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb48759-262c-4358-b867-a21814aa742e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:11.661010 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb48759-262c-4358-b867-a21814aa742e service placement] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:11.698713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 846/1706] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:11.700048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 861/1707] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:11.702762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a877af7-7df3-4a80-8a6e-61c6d3ff165d None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:11.703257 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a877af7-7df3-4a80-8a6e-61c6d3ff165d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:11.703639 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a877af7-7df3-4a80-8a6e-61c6d3ff165d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:11.744170 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-3a877af7-7df3-4a80-8a6e-61c6d3ff165d None None] MFA Rules not processed for user `0d5336bcea514f378e06737680ae9bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:11.745795 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-3a877af7-7df3-4a80-8a6e-61c6d3ff165d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:11.801672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:11.908868 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 847/1708] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:11 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:12.123915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb48759-262c-4358-b867-a21814aa742e service placement] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:12.133756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-0fb48759-262c-4358-b867-a21814aa742e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:12.167129 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb48759-262c-4358-b867-a21814aa742e service placement] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:12.205890 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e053143-9d97-4c65-baf6-442101b1d775 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:12.209497 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e053143-9d97-4c65-baf6-442101b1d775 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:12.209784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e053143-9d97-4c65-baf6-442101b1d775 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:12.210061 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e053143-9d97-4c65-baf6-442101b1d775 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:12.248487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e053143-9d97-4c65-baf6-442101b1d775 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:12.252691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e053143-9d97-4c65-baf6-442101b1d775 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:12.328920 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 862/1709] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:25:12.432155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee02c0-82d4-4569-97bb-2da989d202c1 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:12.432672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee02c0-82d4-4569-97bb-2da989d202c1 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:12.432672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee02c0-82d4-4569-97bb-2da989d202c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:12.506362 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-28ee02c0-82d4-4569-97bb-2da989d202c1 None None] MFA Rules not processed for user `d441bbdb43c5403abf8480146bc15753`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:12.507717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-28ee02c0-82d4-4569-97bb-2da989d202c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:12.649876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e053143-9d97-4c65-baf6-442101b1d775 service nova] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:12.654653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-1e053143-9d97-4c65-baf6-442101b1d775 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:12.690898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e053143-9d97-4c65-baf6-442101b1d775 service nova] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:12.717974 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897f8d0-9115-4a08-a664-0f5fd50ea433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:12.719660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897f8d0-9115-4a08-a664-0f5fd50ea433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:12.719952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897f8d0-9115-4a08-a664-0f5fd50ea433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:12.720237 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897f8d0-9115-4a08-a664-0f5fd50ea433 admin admin] PATH_INFO: `/v3/projects/293acde59f5a418b8da3e49a9ee76893` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:12.721544 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897f8d0-9115-4a08-a664-0f5fd50ea433 admin admin] RBAC: Authorizing `identity:delete_project(project_id=293acde59f5a418b8da3e49a9ee76893)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:12.723984 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897f8d0-9115-4a08-a664-0f5fd50ea433 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:12.838170 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 848/1710] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:12.910579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e897f8d0-9115-4a08-a664-0f5fd50ea433 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '293acde59f5a418b8da3e49a9ee76893', 'request_id': 'req-e897f8d0-9115-4a08-a664-0f5fd50ea433'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:12.936401 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e897f8d0-9115-4a08-a664-0f5fd50ea433 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '293acde59f5a418b8da3e49a9ee76893', 'request_id': 'req-e897f8d0-9115-4a08-a664-0f5fd50ea433'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:13.016313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e897f8d0-9115-4a08-a664-0f5fd50ea433 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '293acde59f5a418b8da3e49a9ee76893', 'request_id': 'req-e897f8d0-9115-4a08-a664-0f5fd50ea433'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:13.047559 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e897f8d0-9115-4a08-a664-0f5fd50ea433 admin admin] The token cache is being invalidate because project 293acde59f5a418b8da3e49a9ee76893 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:13.047898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e897f8d0-9115-4a08-a664-0f5fd50ea433 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:13.057719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e897f8d0-9115-4a08-a664-0f5fd50ea433 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:13.068871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 863/1711] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:12 2022] DELETE /identity/v3/projects/293acde59f5a418b8da3e49a9ee76893 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:13.069036 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee02c0-82d4-4569-97bb-2da989d202c1 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:13.084091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee02c0-82d4-4569-97bb-2da989d202c1 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:13.267530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0cf5e-96a9-49e0-b4b2-9fb612d71850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:13.268187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0cf5e-96a9-49e0-b4b2-9fb612d71850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:13.268471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0cf5e-96a9-49e0-b4b2-9fb612d71850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:13.268746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0cf5e-96a9-49e0-b4b2-9fb612d71850 admin admin] PATH_INFO: `/v3/users/384f8c00cb2648a8879bbeddae5c136a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:13.274244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0cf5e-96a9-49e0-b4b2-9fb612d71850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=384f8c00cb2648a8879bbeddae5c136a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:13.274244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0cf5e-96a9-49e0-b4b2-9fb612d71850 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:13.533920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-cbf0cf5e-96a9-49e0-b4b2-9fb612d71850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '384f8c00cb2648a8879bbeddae5c136a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:13.615950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-cbf0cf5e-96a9-49e0-b4b2-9fb612d71850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '384f8c00cb2648a8879bbeddae5c136a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:13.616755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-cbf0cf5e-96a9-49e0-b4b2-9fb612d71850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '384f8c00cb2648a8879bbeddae5c136a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:13.646519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-cbf0cf5e-96a9-49e0-b4b2-9fb612d71850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '384f8c00cb2648a8879bbeddae5c136a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:13.647770 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-cbf0cf5e-96a9-49e0-b4b2-9fb612d71850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '384f8c00cb2648a8879bbeddae5c136a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:13.655597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:13.680205 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 849/1712] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:13 2022] DELETE /identity/v3/users/384f8c00cb2648a8879bbeddae5c136a => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:15.047493 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897f8d0-9115-4a08-a664-0f5fd50ea433 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:15.054345 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e897f8d0-9115-4a08-a664-0f5fd50ea433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:15.105681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897f8d0-9115-4a08-a664-0f5fd50ea433 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:15.212974 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:15.263945 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9c240-22e5-4736-9971-a05543e14e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:15.267055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9c240-22e5-4736-9971-a05543e14e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:15.267962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9c240-22e5-4736-9971-a05543e14e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:15.270177 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9c240-22e5-4736-9971-a05543e14e3b admin admin] PATH_INFO: `/v3/projects/de0f3169682546cbb9e326cd899a4460` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:15.285629 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9c240-22e5-4736-9971-a05543e14e3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=de0f3169682546cbb9e326cd899a4460)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:15.313113 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9c240-22e5-4736-9971-a05543e14e3b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:15.568281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6eb9c240-22e5-4736-9971-a05543e14e3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de0f3169682546cbb9e326cd899a4460', 'request_id': 'req-6eb9c240-22e5-4736-9971-a05543e14e3b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:15.592972 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6eb9c240-22e5-4736-9971-a05543e14e3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de0f3169682546cbb9e326cd899a4460', 'request_id': 'req-6eb9c240-22e5-4736-9971-a05543e14e3b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:15.661568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6eb9c240-22e5-4736-9971-a05543e14e3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de0f3169682546cbb9e326cd899a4460', 'request_id': 'req-6eb9c240-22e5-4736-9971-a05543e14e3b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:15.706619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6eb9c240-22e5-4736-9971-a05543e14e3b admin admin] The token cache is being invalidate because project de0f3169682546cbb9e326cd899a4460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:15.707852 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6eb9c240-22e5-4736-9971-a05543e14e3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:15.712620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6eb9c240-22e5-4736-9971-a05543e14e3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:15.721912 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 864/1713] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:15 2022] DELETE /identity/v3/projects/de0f3169682546cbb9e326cd899a4460 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:15.725137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1344ae4-e412-4dc6-ad09-e69dd262d1a4 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:15.729275 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1344ae4-e412-4dc6-ad09-e69dd262d1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:15.729275 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1344ae4-e412-4dc6-ad09-e69dd262d1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:15.795093 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-c1344ae4-e412-4dc6-ad09-e69dd262d1a4 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:15.796719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c1344ae4-e412-4dc6-ad09-e69dd262d1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:15.837762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9c240-22e5-4736-9971-a05543e14e3b admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:15.839492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6eb9c240-22e5-4736-9971-a05543e14e3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:16.022944 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 850/1714] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:16.023525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9c240-22e5-4736-9971-a05543e14e3b admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:16.041298 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb274cf-b6f9-4d6b-b300-715538caae94 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:16.041864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb274cf-b6f9-4d6b-b300-715538caae94 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:16.042417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb274cf-b6f9-4d6b-b300-715538caae94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:16.043672 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199de8ac-abdb-4fb9-8c7b-7d541b1bce45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:16.044340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199de8ac-abdb-4fb9-8c7b-7d541b1bce45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:16.045272 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199de8ac-abdb-4fb9-8c7b-7d541b1bce45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:16.045725 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199de8ac-abdb-4fb9-8c7b-7d541b1bce45 admin admin] PATH_INFO: `/v3/users/6b94b7297d0f4676a6272ea0c6e99895` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:16.049395 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199de8ac-abdb-4fb9-8c7b-7d541b1bce45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b94b7297d0f4676a6272ea0c6e99895)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:16.058831 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199de8ac-abdb-4fb9-8c7b-7d541b1bce45 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:16.096829 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-7eb274cf-b6f9-4d6b-b300-715538caae94 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:16.099458 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-7eb274cf-b6f9-4d6b-b300-715538caae94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:16.117668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 851/1715] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:16.121741 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb274cf-b6f9-4d6b-b300-715538caae94 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:16.140614 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb274cf-b6f9-4d6b-b300-715538caae94 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:16.150391 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04562df9-f21e-4dc0-9916-6b6de1b95f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:16.151310 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04562df9-f21e-4dc0-9916-6b6de1b95f4e admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:16.151930 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04562df9-f21e-4dc0-9916-6b6de1b95f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:16.152556 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04562df9-f21e-4dc0-9916-6b6de1b95f4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:16.153391 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04562df9-f21e-4dc0-9916-6b6de1b95f4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:25:16.153856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04562df9-f21e-4dc0-9916-6b6de1b95f4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:16.160142 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04562df9-f21e-4dc0-9916-6b6de1b95f4e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:16.196645 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 852/1716] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:25:16.199532 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04562df9-f21e-4dc0-9916-6b6de1b95f4e admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:16.213065 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04562df9-f21e-4dc0-9916-6b6de1b95f4e admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:16.224903 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0de29-3057-4250-9c05-c4d31bd0911f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:16.227989 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0de29-3057-4250-9c05-c4d31bd0911f admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:16.228279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0de29-3057-4250-9c05-c4d31bd0911f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:16.229747 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0de29-3057-4250-9c05-c4d31bd0911f admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:16.231538 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0de29-3057-4250-9c05-c4d31bd0911f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:16.236061 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0de29-3057-4250-9c05-c4d31bd0911f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:16.370540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-199de8ac-abdb-4fb9-8c7b-7d541b1bce45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b94b7297d0f4676a6272ea0c6e99895'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:16.386862 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 853/1717] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:16 2022] POST /identity/v3/projects => generated 401 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:16.392170 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0de29-3057-4250-9c05-c4d31bd0911f admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:16.410458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-199de8ac-abdb-4fb9-8c7b-7d541b1bce45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b94b7297d0f4676a6272ea0c6e99895'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:16.523027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-199de8ac-abdb-4fb9-8c7b-7d541b1bce45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b94b7297d0f4676a6272ea0c6e99895'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:16.543357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-199de8ac-abdb-4fb9-8c7b-7d541b1bce45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b94b7297d0f4676a6272ea0c6e99895'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:16.544410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-199de8ac-abdb-4fb9-8c7b-7d541b1bce45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b94b7297d0f4676a6272ea0c6e99895'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:16.544868 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0de29-3057-4250-9c05-c4d31bd0911f admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:16.555022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:16.563606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 865/1718] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:15 2022] DELETE /identity/v3/users/6b94b7297d0f4676a6272ea0c6e99895 => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:16.568739 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ce95c-86c7-430a-997a-3d4e9f7a1fdb None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:16.569270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ce95c-86c7-430a-997a-3d4e9f7a1fdb None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:16.572309 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ce95c-86c7-430a-997a-3d4e9f7a1fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:16.573743 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1bd80-df6a-4870-97a4-cf0d40ebce18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:16.574382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1bd80-df6a-4870-97a4-cf0d40ebce18 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:16.575556 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1bd80-df6a-4870-97a4-cf0d40ebce18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:16.576558 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1bd80-df6a-4870-97a4-cf0d40ebce18 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:16.577912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1bd80-df6a-4870-97a4-cf0d40ebce18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:16.580420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1bd80-df6a-4870-97a4-cf0d40ebce18 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:16.606752 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-2b5ce95c-86c7-430a-997a-3d4e9f7a1fdb None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:16.609010 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-2b5ce95c-86c7-430a-997a-3d4e9f7a1fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:16.746483 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 854/1719] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:16 2022] POST /identity/v3/users => generated 301 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:16.751574 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1bd80-df6a-4870-97a4-cf0d40ebce18 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:16.753347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-68d1bd80-df6a-4870-97a4-cf0d40ebce18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:16.802922 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 866/1720] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:16.809416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ce95c-86c7-430a-997a-3d4e9f7a1fdb None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:16.811862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-2b5ce95c-86c7-430a-997a-3d4e9f7a1fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:16.820864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1bd80-df6a-4870-97a4-cf0d40ebce18 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:16.840078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e1617-fb6a-4cc2-b288-c8f634f4cf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:16.840703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e1617-fb6a-4cc2-b288-c8f634f4cf67 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:16.841337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e1617-fb6a-4cc2-b288-c8f634f4cf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:16.841983 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e1617-fb6a-4cc2-b288-c8f634f4cf67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:16.842819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e1617-fb6a-4cc2-b288-c8f634f4cf67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:16.844636 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e1617-fb6a-4cc2-b288-c8f634f4cf67 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:16.866775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 855/1721] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:16.869601 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e1617-fb6a-4cc2-b288-c8f634f4cf67 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:16.885002 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e1617-fb6a-4cc2-b288-c8f634f4cf67 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:16.910494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec3683-446f-498a-9e6b-06e82f0afeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:16.913543 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec3683-446f-498a-9e6b-06e82f0afeef admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:16.916097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec3683-446f-498a-9e6b-06e82f0afeef admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:16.916344 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec3683-446f-498a-9e6b-06e82f0afeef admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:16.921480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec3683-446f-498a-9e6b-06e82f0afeef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:16.929445 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec3683-446f-498a-9e6b-06e82f0afeef admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:16.952474 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ce95c-86c7-430a-997a-3d4e9f7a1fdb None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:16.961738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5b29c-784e-42cc-bbb1-c3242c6369f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:16.962691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5b29c-784e-42cc-bbb1-c3242c6369f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:16.963103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5b29c-784e-42cc-bbb1-c3242c6369f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:16.963891 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 856/1722] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:25:16.965675 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5b29c-784e-42cc-bbb1-c3242c6369f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:16.966440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec3683-446f-498a-9e6b-06e82f0afeef admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:17.009423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec3683-446f-498a-9e6b-06e82f0afeef admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:17.012454 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5b29c-784e-42cc-bbb1-c3242c6369f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:17.019173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99843f-e528-49c2-92b8-baceabe87a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:17.019607 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5b29c-784e-42cc-bbb1-c3242c6369f1 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:17.020566 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99843f-e528-49c2-92b8-baceabe87a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:17.021241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99843f-e528-49c2-92b8-baceabe87a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:17.022209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99843f-e528-49c2-92b8-baceabe87a3f admin admin] PATH_INFO: `/v3/projects/85e9ac8822664660b78688c2d8125ea8/users/5feea897276749ebae5ef50c4bdbb2dd/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:17.045862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 867/1723] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:25:17.054611 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a99843f-e528-49c2-92b8-baceabe87a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=85e9ac8822664660b78688c2d8125ea8, user_id=5feea897276749ebae5ef50c4bdbb2dd, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:17.058033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a99843f-e528-49c2-92b8-baceabe87a3f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:17.064276 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:17.201197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 857/1724] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:16 2022] PUT /identity/v3/projects/85e9ac8822664660b78688c2d8125ea8/users/5feea897276749ebae5ef50c4bdbb2dd/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:17.217989 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46740f-0d70-4ff3-97bc-53214c5ae52d None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:17.218462 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46740f-0d70-4ff3-97bc-53214c5ae52d None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:17.218682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46740f-0d70-4ff3-97bc-53214c5ae52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:17.285842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-8d46740f-0d70-4ff3-97bc-53214c5ae52d None None] MFA Rules not processed for user `5feea897276749ebae5ef50c4bdbb2dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:17.287257 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8d46740f-0d70-4ff3-97bc-53214c5ae52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:17.411431 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 868/1725] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:17 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:17.417372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99843f-e528-49c2-92b8-baceabe87a3f admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:17.526588 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99843f-e528-49c2-92b8-baceabe87a3f admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:17.535339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60a79d-0020-4f8b-956f-4b4d078e9266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:17.535966 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60a79d-0020-4f8b-956f-4b4d078e9266 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:17.536395 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60a79d-0020-4f8b-956f-4b4d078e9266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:17.538168 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60a79d-0020-4f8b-956f-4b4d078e9266 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:17.538168 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60a79d-0020-4f8b-956f-4b4d078e9266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:17.540020 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60a79d-0020-4f8b-956f-4b4d078e9266 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:17.647874 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78011958-c125-4241-a34c-b49e41b0a8c8 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:17.648165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78011958-c125-4241-a34c-b49e41b0a8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:17.648440 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78011958-c125-4241-a34c-b49e41b0a8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:17.672937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 858/1726] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:17 2022] POST /identity/v3/projects => generated 399 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:17.677265 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60a79d-0020-4f8b-956f-4b4d078e9266 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:17.684375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-78011958-c125-4241-a34c-b49e41b0a8c8 None None] MFA Rules not processed for user `8bc4801f1f674f55b57d49e5f09205ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:17.692622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-78011958-c125-4241-a34c-b49e41b0a8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:17.775409 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 869/1727] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:17.776450 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60a79d-0020-4f8b-956f-4b4d078e9266 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:17.782377 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aaacb3-23a2-42c4-b4f4-cacc3620bb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:17.783152 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaacb3-23a2-42c4-b4f4-cacc3620bb75 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:17.783508 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaacb3-23a2-42c4-b4f4-cacc3620bb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:17.783863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaacb3-23a2-42c4-b4f4-cacc3620bb75 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:17.784639 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aaacb3-23a2-42c4-b4f4-cacc3620bb75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:17.786920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aaacb3-23a2-42c4-b4f4-cacc3620bb75 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:17.795216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78011958-c125-4241-a34c-b49e41b0a8c8 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:17.799911 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-78011958-c125-4241-a34c-b49e41b0a8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:17.815337 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78011958-c125-4241-a34c-b49e41b0a8c8 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:17.820237 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99435514-3a08-4066-8712-57dfbb822d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:17.820887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99435514-3a08-4066-8712-57dfbb822d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:17.825673 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99435514-3a08-4066-8712-57dfbb822d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:17.825673 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99435514-3a08-4066-8712-57dfbb822d20 admin admin] PATH_INFO: `/v3/projects/b035a36532cd45689246f990030a3a76` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:17.825673 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99435514-3a08-4066-8712-57dfbb822d20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b035a36532cd45689246f990030a3a76)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:17.831988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99435514-3a08-4066-8712-57dfbb822d20 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:17.844127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:17.879173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 859/1728] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:17 2022] POST /identity/v3/users => generated 306 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:25:17.883177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aaacb3-23a2-42c4-b4f4-cacc3620bb75 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:17.893072 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aaacb3-23a2-42c4-b4f4-cacc3620bb75 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:17.897853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97293634-27b9-489d-98fd-04386fb16944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:17.898430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97293634-27b9-489d-98fd-04386fb16944 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:17.898703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97293634-27b9-489d-98fd-04386fb16944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:17.899005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97293634-27b9-489d-98fd-04386fb16944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:17.899639 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97293634-27b9-489d-98fd-04386fb16944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:17.901231 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97293634-27b9-489d-98fd-04386fb16944 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:17.912733 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 860/1729] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:25:17.916749 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97293634-27b9-489d-98fd-04386fb16944 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:17.928809 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97293634-27b9-489d-98fd-04386fb16944 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:17.934008 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a38b5-0311-4ab6-808c-5bd4356b64f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:17.934919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a38b5-0311-4ab6-808c-5bd4356b64f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:17.935329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a38b5-0311-4ab6-808c-5bd4356b64f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:17.935703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a38b5-0311-4ab6-808c-5bd4356b64f4 admin admin] PATH_INFO: `/v3/projects/14d845bf302445b89c3048e4aeebba30/users/499f5a13a2394570982138745db67308/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:17.955780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a38b5-0311-4ab6-808c-5bd4356b64f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14d845bf302445b89c3048e4aeebba30, user_id=499f5a13a2394570982138745db67308, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:17.958491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a38b5-0311-4ab6-808c-5bd4356b64f4 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:18.074965 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 861/1730] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:17 2022] PUT /identity/v3/projects/14d845bf302445b89c3048e4aeebba30/users/499f5a13a2394570982138745db67308/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:18.077328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d67a8-bb62-4e50-9023-f6aade732167 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:18.077697 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d67a8-bb62-4e50-9023-f6aade732167 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:18.077980 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d67a8-bb62-4e50-9023-f6aade732167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:18.081430 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-99435514-3a08-4066-8712-57dfbb822d20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b035a36532cd45689246f990030a3a76', 'request_id': 'req-99435514-3a08-4066-8712-57dfbb822d20'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:18.090798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-99435514-3a08-4066-8712-57dfbb822d20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b035a36532cd45689246f990030a3a76', 'request_id': 'req-99435514-3a08-4066-8712-57dfbb822d20'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:18.104398 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-0c1d67a8-bb62-4e50-9023-f6aade732167 None None] MFA Rules not processed for user `0d5336bcea514f378e06737680ae9bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:18.105837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-0c1d67a8-bb62-4e50-9023-f6aade732167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:18.141517 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:18.164719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-99435514-3a08-4066-8712-57dfbb822d20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b035a36532cd45689246f990030a3a76', 'request_id': 'req-99435514-3a08-4066-8712-57dfbb822d20'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:18.181040 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 862/1731] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:18.181040 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d67a8-bb62-4e50-9023-f6aade732167 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:18.188303 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-99435514-3a08-4066-8712-57dfbb822d20 admin admin] The token cache is being invalidate because project b035a36532cd45689246f990030a3a76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:18.188567 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-99435514-3a08-4066-8712-57dfbb822d20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:18.189428 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-99435514-3a08-4066-8712-57dfbb822d20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:18.193352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 870/1732] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:17 2022] DELETE /identity/v3/projects/b035a36532cd45689246f990030a3a76 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:18.199519 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fd884-4936-401d-ba51-83ee3eea3273 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:18.200013 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fd884-4936-401d-ba51-83ee3eea3273 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:18.200368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fd884-4936-401d-ba51-83ee3eea3273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:18.238612 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-319fd884-4936-401d-ba51-83ee3eea3273 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:18.243182 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-319fd884-4936-401d-ba51-83ee3eea3273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:18.340660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d67a8-bb62-4e50-9023-f6aade732167 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:18.349235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41562ec-ec90-43e2-b003-94e80ddda682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:18.350154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41562ec-ec90-43e2-b003-94e80ddda682 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:18.351146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41562ec-ec90-43e2-b003-94e80ddda682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:18.351549 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41562ec-ec90-43e2-b003-94e80ddda682 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:18.353235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41562ec-ec90-43e2-b003-94e80ddda682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:18.355132 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41562ec-ec90-43e2-b003-94e80ddda682 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:18.388044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 871/1733] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:25:18.389543 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 863/1734] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:18.390602 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fd884-4936-401d-ba51-83ee3eea3273 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:18.399789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1ba22-23d0-4c36-aabc-27c31fd18c5a None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:18.400251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1ba22-23d0-4c36-aabc-27c31fd18c5a None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:18.400705 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1ba22-23d0-4c36-aabc-27c31fd18c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:18.407443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-319fd884-4936-401d-ba51-83ee3eea3273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:18.425153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-01b1ba22-23d0-4c36-aabc-27c31fd18c5a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:18.427896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-01b1ba22-23d0-4c36-aabc-27c31fd18c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:18.444345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 864/1735] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:18.445327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1ba22-23d0-4c36-aabc-27c31fd18c5a None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:18.447791 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fd884-4936-401d-ba51-83ee3eea3273 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:18.456701 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1a319-69cf-447f-bdcf-87f3f835f44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:18.457804 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1a319-69cf-447f-bdcf-87f3f835f44a admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:18.460740 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1a319-69cf-447f-bdcf-87f3f835f44a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:18.461139 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1a319-69cf-447f-bdcf-87f3f835f44a admin admin] PATH_INFO: `/v3/projects/14d845bf302445b89c3048e4aeebba30/users/499f5a13a2394570982138745db67308/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:18.462432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1ba22-23d0-4c36-aabc-27c31fd18c5a None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:18.468348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1a319-69cf-447f-bdcf-87f3f835f44a admin admin] RBAC: Authorizing `identity:create_grant(project_id=14d845bf302445b89c3048e4aeebba30, user_id=499f5a13a2394570982138745db67308, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:18.475422 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1a319-69cf-447f-bdcf-87f3f835f44a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:18.499052 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8800e6-efb3-4c95-8fb9-09ae5305407d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:18.500140 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8800e6-efb3-4c95-8fb9-09ae5305407d admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:18.500533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8800e6-efb3-4c95-8fb9-09ae5305407d admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:18.500967 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8800e6-efb3-4c95-8fb9-09ae5305407d admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:18.501749 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8800e6-efb3-4c95-8fb9-09ae5305407d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:25:18.502212 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8800e6-efb3-4c95-8fb9-09ae5305407d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:18.506927 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8800e6-efb3-4c95-8fb9-09ae5305407d admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:18.515281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 872/1736] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:18 2022] PUT /identity/v3/projects/14d845bf302445b89c3048e4aeebba30/users/499f5a13a2394570982138745db67308/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:18.517340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b9ea1-e548-4f4b-b695-257fc7d2e463 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:18.522203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b9ea1-e548-4f4b-b695-257fc7d2e463 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:18.522203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b9ea1-e548-4f4b-b695-257fc7d2e463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:18.526863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 865/1737] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:25:18.533241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8800e6-efb3-4c95-8fb9-09ae5305407d admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:18.565519 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-bf9b9ea1-e548-4f4b-b695-257fc7d2e463 None None] MFA Rules not processed for user `499f5a13a2394570982138745db67308`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:18.569458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-bf9b9ea1-e548-4f4b-b695-257fc7d2e463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:18.577274 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8800e6-efb3-4c95-8fb9-09ae5305407d admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:18.601225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:18.714929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 873/1738] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:18 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:18.724029 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b9ea1-e548-4f4b-b695-257fc7d2e463 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:18.726502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-bf9b9ea1-e548-4f4b-b695-257fc7d2e463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:18.736466 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba1376-3aa5-4842-a044-62c9d2d9b814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:18.737792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba1376-3aa5-4842-a044-62c9d2d9b814 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:18.737915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba1376-3aa5-4842-a044-62c9d2d9b814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:18.738212 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba1376-3aa5-4842-a044-62c9d2d9b814 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:18.739293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba1376-3aa5-4842-a044-62c9d2d9b814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:18.741039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba1376-3aa5-4842-a044-62c9d2d9b814 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:18.821003 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b9ea1-e548-4f4b-b695-257fc7d2e463 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:18.827254 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868dfaa3-820d-4d0f-86ec-9fe64c72286f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:18.827254 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dfaa3-820d-4d0f-86ec-9fe64c72286f service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:18.827254 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dfaa3-820d-4d0f-86ec-9fe64c72286f service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:18.827254 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dfaa3-820d-4d0f-86ec-9fe64c72286f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:18.844262 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 866/1739] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:18 2022] POST /identity/v3/projects => generated 367 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:18.848759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba1376-3aa5-4842-a044-62c9d2d9b814 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:18.851710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868dfaa3-820d-4d0f-86ec-9fe64c72286f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:18.853814 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868dfaa3-820d-4d0f-86ec-9fe64c72286f service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:18.891077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 874/1740] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:18 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:18.975081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:19.006822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba1376-3aa5-4842-a044-62c9d2d9b814 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:19.024548 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28c101-ecd3-4385-8a83-c8c6a7887599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:19.031975 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28c101-ecd3-4385-8a83-c8c6a7887599 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:19.031975 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28c101-ecd3-4385-8a83-c8c6a7887599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:19.031975 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28c101-ecd3-4385-8a83-c8c6a7887599 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:19.031975 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28c101-ecd3-4385-8a83-c8c6a7887599 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:19.031975 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28c101-ecd3-4385-8a83-c8c6a7887599 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:19.161270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 867/1741] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:18 2022] POST /identity/v3/users => generated 284 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:19.166009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868dfaa3-820d-4d0f-86ec-9fe64c72286f service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:19.168930 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28c101-ecd3-4385-8a83-c8c6a7887599 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:19.183981 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868dfaa3-820d-4d0f-86ec-9fe64c72286f service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:19.195921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3018a9b-a1d3-4585-b08e-8b69add74e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:19.205500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3018a9b-a1d3-4585-b08e-8b69add74e3c admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:19.205500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3018a9b-a1d3-4585-b08e-8b69add74e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:19.205500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3018a9b-a1d3-4585-b08e-8b69add74e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:19.205500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3018a9b-a1d3-4585-b08e-8b69add74e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:19.205500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3018a9b-a1d3-4585-b08e-8b69add74e3c admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:19.236447 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 875/1742] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:19.239457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3018a9b-a1d3-4585-b08e-8b69add74e3c admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:19.268686 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3018a9b-a1d3-4585-b08e-8b69add74e3c admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:19.283074 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd63aa2-7fb0-4dbb-9314-136505c769fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:19.283714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd63aa2-7fb0-4dbb-9314-136505c769fa admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:19.283999 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd63aa2-7fb0-4dbb-9314-136505c769fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:19.284296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd63aa2-7fb0-4dbb-9314-136505c769fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:19.285008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd63aa2-7fb0-4dbb-9314-136505c769fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:19.286662 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd63aa2-7fb0-4dbb-9314-136505c769fa admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:19.297398 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28c101-ecd3-4385-8a83-c8c6a7887599 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:19.302740 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c58cc-ae68-43d4-9ae1-c68cfd89d7ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:19.303777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c58cc-ae68-43d4-9ae1-c68cfd89d7ab service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:19.304175 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c58cc-ae68-43d4-9ae1-c68cfd89d7ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:19.304821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c58cc-ae68-43d4-9ae1-c68cfd89d7ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:19.308874 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-634c58cc-ae68-43d4-9ae1-c68cfd89d7ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:19.331312 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 876/1743] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:25:19.332295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd63aa2-7fb0-4dbb-9314-136505c769fa admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:19.366239 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd63aa2-7fb0-4dbb-9314-136505c769fa admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:19.379936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e130c8-ac37-4705-b0b7-fa451d629e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:19.381960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e130c8-ac37-4705-b0b7-fa451d629e6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:19.382313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e130c8-ac37-4705-b0b7-fa451d629e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:19.382661 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e130c8-ac37-4705-b0b7-fa451d629e6d admin admin] PATH_INFO: `/v3/projects/d5fb9d6a11054d3ba63ae1d8e4a21c62/users/a7fafa17c696443dab0cf762c73a4ad4/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:19.422127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e130c8-ac37-4705-b0b7-fa451d629e6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5fb9d6a11054d3ba63ae1d8e4a21c62, user_id=a7fafa17c696443dab0cf762c73a4ad4, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:19.426654 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e130c8-ac37-4705-b0b7-fa451d629e6d admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:19.485528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c58cc-ae68-43d4-9ae1-c68cfd89d7ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:19.487197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c58cc-ae68-43d4-9ae1-c68cfd89d7ab service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:19.502805 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 877/1744] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:19 2022] PUT /identity/v3/projects/d5fb9d6a11054d3ba63ae1d8e4a21c62/users/a7fafa17c696443dab0cf762c73a4ad4/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:19.516512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a44411-89b2-4f80-a813-1a4b683a022a None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:19.517025 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a44411-89b2-4f80-a813-1a4b683a022a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:19.527063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a44411-89b2-4f80-a813-1a4b683a022a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:19.534913 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 868/1745] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:19 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:25:19.666651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-44a44411-89b2-4f80-a813-1a4b683a022a None None] MFA Rules not processed for user `a7fafa17c696443dab0cf762c73a4ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:19.670206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-44a44411-89b2-4f80-a813-1a4b683a022a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:19.759700 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:19.773165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 878/1746] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:19.790662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c58cc-ae68-43d4-9ae1-c68cfd89d7ab service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:19.793434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-634c58cc-ae68-43d4-9ae1-c68cfd89d7ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:19.810507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c58cc-ae68-43d4-9ae1-c68cfd89d7ab service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:19.886198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82650ff1-28fe-48cd-b26e-337cceea3e78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:19.886833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82650ff1-28fe-48cd-b26e-337cceea3e78 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:19.887219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82650ff1-28fe-48cd-b26e-337cceea3e78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:19.887676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82650ff1-28fe-48cd-b26e-337cceea3e78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:19.898092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82650ff1-28fe-48cd-b26e-337cceea3e78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:19.899591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82650ff1-28fe-48cd-b26e-337cceea3e78 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:19.912349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 869/1747] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:21.860572 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720569e-f8a2-4bb7-84ff-c6d3e2b651c5 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:21.860572 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720569e-f8a2-4bb7-84ff-c6d3e2b651c5 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:21.860572 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720569e-f8a2-4bb7-84ff-c6d3e2b651c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:21.880767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801140-812f-4365-8d3d-a8f7e28f5a0b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:21.881231 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801140-812f-4365-8d3d-a8f7e28f5a0b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:21.881617 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801140-812f-4365-8d3d-a8f7e28f5a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:21.953723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-3720569e-f8a2-4bb7-84ff-c6d3e2b651c5 None None] MFA Rules not processed for user `b7a51545146846cc8284b7044b32198a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:21.958338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3720569e-f8a2-4bb7-84ff-c6d3e2b651c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:22.039648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:22.050180 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-c3801140-812f-4365-8d3d-a8f7e28f5a0b None None] MFA Rules not processed for user `1ee990b1658e49cc8bc2fe15e7dc36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:22.054264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c3801140-812f-4365-8d3d-a8f7e28f5a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:22.167648 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 879/1748] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Aug 31 19:25:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:25:22.226484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720569e-f8a2-4bb7-84ff-c6d3e2b651c5 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:22.239877 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3720569e-f8a2-4bb7-84ff-c6d3e2b651c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:22.311151 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720569e-f8a2-4bb7-84ff-c6d3e2b651c5 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:22.333140 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470f796-2bb3-487d-ba15-8002bec2cd3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:22.334559 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470f796-2bb3-487d-ba15-8002bec2cd3c service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:22.334852 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470f796-2bb3-487d-ba15-8002bec2cd3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:22.336254 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470f796-2bb3-487d-ba15-8002bec2cd3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:22.344299 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470f796-2bb3-487d-ba15-8002bec2cd3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:22.360381 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470f796-2bb3-487d-ba15-8002bec2cd3c service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:22.373435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 870/1749] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 31 19:25:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:25:22.403440 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 880/1750] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:22.717071 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3801140-812f-4365-8d3d-a8f7e28f5a0b None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:22.719990 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c3801140-812f-4365-8d3d-a8f7e28f5a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:22.914176 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3801140-812f-4365-8d3d-a8f7e28f5a0b None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:22.923005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61db5e-7455-4793-9f5b-d0c03ba43360 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25ed58369ffd46e8b7fa1880662926e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:22.925113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61db5e-7455-4793-9f5b-d0c03ba43360 service swift] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:22.925481 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61db5e-7455-4793-9f5b-d0c03ba43360 service swift] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:22.925827 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61db5e-7455-4793-9f5b-d0c03ba43360 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:22.938497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61db5e-7455-4793-9f5b-d0c03ba43360 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:22.940294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61db5e-7455-4793-9f5b-d0c03ba43360 service swift] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:22.969742 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 871/1751] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 31 19:25:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:25:25.418634 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470f796-2bb3-487d-ba15-8002bec2cd3c service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:25.420882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3470f796-2bb3-487d-ba15-8002bec2cd3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:25.639443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470f796-2bb3-487d-ba15-8002bec2cd3c service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:25.667000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d471c-4bcf-404e-8ddc-df251f3dd8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:25.667955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d471c-4bcf-404e-8ddc-df251f3dd8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:25.668940 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d471c-4bcf-404e-8ddc-df251f3dd8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:25.671058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d471c-4bcf-404e-8ddc-df251f3dd8c0 admin admin] PATH_INFO: `/v3/users/bbb071487d364e708c8ba6e1bb2de8d6` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:25.675633 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d471c-4bcf-404e-8ddc-df251f3dd8c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbb071487d364e708c8ba6e1bb2de8d6)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:25.690983 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d471c-4bcf-404e-8ddc-df251f3dd8c0 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:26.044718 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-141d471c-4bcf-404e-8ddc-df251f3dd8c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbb071487d364e708c8ba6e1bb2de8d6'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:26.102671 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-141d471c-4bcf-404e-8ddc-df251f3dd8c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbb071487d364e708c8ba6e1bb2de8d6'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:26.210335 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-141d471c-4bcf-404e-8ddc-df251f3dd8c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbb071487d364e708c8ba6e1bb2de8d6'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:26.222746 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-141d471c-4bcf-404e-8ddc-df251f3dd8c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbb071487d364e708c8ba6e1bb2de8d6'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:26.223557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-141d471c-4bcf-404e-8ddc-df251f3dd8c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbb071487d364e708c8ba6e1bb2de8d6'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:26.240281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 881/1752] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:25 2022] DELETE /identity/v3/users/bbb071487d364e708c8ba6e1bb2de8d6 => generated 0 bytes in 822 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:26.245651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedee57-9c51-4313-a723-4ee19f5d01c2 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:26.246150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedee57-9c51-4313-a723-4ee19f5d01c2 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:26.246369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedee57-9c51-4313-a723-4ee19f5d01c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:26.283962 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ceedee57-9c51-4313-a723-4ee19f5d01c2 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:26.285270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ceedee57-9c51-4313-a723-4ee19f5d01c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:26.388804 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 872/1753] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:26.451620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d471c-4bcf-404e-8ddc-df251f3dd8c0 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:26.454779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-141d471c-4bcf-404e-8ddc-df251f3dd8c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:26.487960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:26.592625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d471c-4bcf-404e-8ddc-df251f3dd8c0 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:26.597012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2df87-0c8b-482c-9353-14b8bb7ac216 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:26.600171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2df87-0c8b-482c-9353-14b8bb7ac216 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:26.600171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2df87-0c8b-482c-9353-14b8bb7ac216 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:26.600171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2df87-0c8b-482c-9353-14b8bb7ac216 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:26.613541 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2df87-0c8b-482c-9353-14b8bb7ac216 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:26.639986 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2df87-0c8b-482c-9353-14b8bb7ac216 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:26.678465 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 882/1754] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:26.813874 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696d5d9-0922-4f88-b7bf-41d89a306cb5 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:26.814063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696d5d9-0922-4f88-b7bf-41d89a306cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:26.814362 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696d5d9-0922-4f88-b7bf-41d89a306cb5 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:26.816181 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 873/1755] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 31 19:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:25:26.829337 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85130d2e-3c36-4d02-96e2-046e995cd38d None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:26.829629 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85130d2e-3c36-4d02-96e2-046e995cd38d None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:26.829913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85130d2e-3c36-4d02-96e2-046e995cd38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:26.859190 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-85130d2e-3c36-4d02-96e2-046e995cd38d None None] MFA Rules not processed for user `d461422ce98442bcbe6b254a8dad5e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:26.860499 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-85130d2e-3c36-4d02-96e2-046e995cd38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:27.021269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 883/1756] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 31 19:25:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:27.386112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696d5d9-0922-4f88-b7bf-41d89a306cb5 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:27.389681 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e696d5d9-0922-4f88-b7bf-41d89a306cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:27.448379 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696d5d9-0922-4f88-b7bf-41d89a306cb5 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:27.468426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17baed-fcc3-4f47-8c98-a510c0117689 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:27.469605 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17baed-fcc3-4f47-8c98-a510c0117689 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:27.470111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17baed-fcc3-4f47-8c98-a510c0117689 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:27.470394 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17baed-fcc3-4f47-8c98-a510c0117689 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:27.485812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17baed-fcc3-4f47-8c98-a510c0117689 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:27.488218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17baed-fcc3-4f47-8c98-a510c0117689 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:27.495329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:27.517657 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 874/1757] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:27.537975 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85130d2e-3c36-4d02-96e2-046e995cd38d None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:27.539465 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-85130d2e-3c36-4d02-96e2-046e995cd38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:27.569187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85130d2e-3c36-4d02-96e2-046e995cd38d None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:27.603749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48959c-dd4d-414f-9e61-b647c7ec3728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:27.610358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48959c-dd4d-414f-9e61-b647c7ec3728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:27.610358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48959c-dd4d-414f-9e61-b647c7ec3728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:27.610358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48959c-dd4d-414f-9e61-b647c7ec3728 admin admin] PATH_INFO: `/v3/users/5feea897276749ebae5ef50c4bdbb2dd` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:27.610358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48959c-dd4d-414f-9e61-b647c7ec3728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5feea897276749ebae5ef50c4bdbb2dd)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:27.611713 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48959c-dd4d-414f-9e61-b647c7ec3728 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:28.042178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ff48959c-dd4d-414f-9e61-b647c7ec3728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5feea897276749ebae5ef50c4bdbb2dd'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:28.087389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ff48959c-dd4d-414f-9e61-b647c7ec3728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5feea897276749ebae5ef50c4bdbb2dd'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:28.175656 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ff48959c-dd4d-414f-9e61-b647c7ec3728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5feea897276749ebae5ef50c4bdbb2dd'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:28.186834 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ff48959c-dd4d-414f-9e61-b647c7ec3728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5feea897276749ebae5ef50c4bdbb2dd'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:28.191146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ff48959c-dd4d-414f-9e61-b647c7ec3728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5feea897276749ebae5ef50c4bdbb2dd'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:28.206201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 884/1758] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:27 2022] DELETE /identity/v3/users/5feea897276749ebae5ef50c4bdbb2dd => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:28.210890 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79574c27-2809-45bc-8090-077a008d5d22 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:28.211279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79574c27-2809-45bc-8090-077a008d5d22 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:28.211630 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79574c27-2809-45bc-8090-077a008d5d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:28.260024 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-79574c27-2809-45bc-8090-077a008d5d22 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:28.261242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-79574c27-2809-45bc-8090-077a008d5d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:28.407500 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 875/1759] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:28.480985 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48959c-dd4d-414f-9e61-b647c7ec3728 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:28.482571 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ff48959c-dd4d-414f-9e61-b647c7ec3728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:28.533703 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:28.623143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48959c-dd4d-414f-9e61-b647c7ec3728 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:28.636387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b803b0d-75c0-40e6-8fc1-5a7f02d3a87e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:28.637467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b803b0d-75c0-40e6-8fc1-5a7f02d3a87e service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:28.637913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b803b0d-75c0-40e6-8fc1-5a7f02d3a87e service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:28.638991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b803b0d-75c0-40e6-8fc1-5a7f02d3a87e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:28.668434 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b803b0d-75c0-40e6-8fc1-5a7f02d3a87e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:28.668919 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b803b0d-75c0-40e6-8fc1-5a7f02d3a87e service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:28.696723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 885/1760] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:28.972205 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79574c27-2809-45bc-8090-077a008d5d22 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:28.974032 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-79574c27-2809-45bc-8090-077a008d5d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:28.979629 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b803b0d-75c0-40e6-8fc1-5a7f02d3a87e service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:28.979629 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3b803b0d-75c0-40e6-8fc1-5a7f02d3a87e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:28.998753 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79574c27-2809-45bc-8090-077a008d5d22 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:29.000779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b803b0d-75c0-40e6-8fc1-5a7f02d3a87e service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:29.006543 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba22e79-2101-4abf-b982-207bbdae5595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:29.007875 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba22e79-2101-4abf-b982-207bbdae5595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:29.008509 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba22e79-2101-4abf-b982-207bbdae5595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:29.009048 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba22e79-2101-4abf-b982-207bbdae5595 admin admin] PATH_INFO: `/v3/users/324faeccb7fa41c093bfae329fd960d0` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:29.011032 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba22e79-2101-4abf-b982-207bbdae5595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=324faeccb7fa41c093bfae329fd960d0)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:29.015706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba22e79-2101-4abf-b982-207bbdae5595 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:29.025322 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9dd25-eae9-464e-8fad-f657a6f95d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:29.025864 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9dd25-eae9-464e-8fad-f657a6f95d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:29.026165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9dd25-eae9-464e-8fad-f657a6f95d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:29.026568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9dd25-eae9-464e-8fad-f657a6f95d70 admin admin] PATH_INFO: `/v3/projects/30121138fee54489bdebebd5345abac5` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:29.027943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a9dd25-eae9-464e-8fad-f657a6f95d70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30121138fee54489bdebebd5345abac5)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:29.030765 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a9dd25-eae9-464e-8fad-f657a6f95d70 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:29.247840 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-80a9dd25-eae9-464e-8fad-f657a6f95d70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30121138fee54489bdebebd5345abac5', 'request_id': 'req-80a9dd25-eae9-464e-8fad-f657a6f95d70'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:29.258072 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-80a9dd25-eae9-464e-8fad-f657a6f95d70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30121138fee54489bdebebd5345abac5', 'request_id': 'req-80a9dd25-eae9-464e-8fad-f657a6f95d70'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:29.397261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-80a9dd25-eae9-464e-8fad-f657a6f95d70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30121138fee54489bdebebd5345abac5', 'request_id': 'req-80a9dd25-eae9-464e-8fad-f657a6f95d70'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:29.420715 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-80a9dd25-eae9-464e-8fad-f657a6f95d70 admin admin] The token cache is being invalidate because project 30121138fee54489bdebebd5345abac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:29.421453 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-80a9dd25-eae9-464e-8fad-f657a6f95d70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:29.441429 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6ba22e79-2101-4abf-b982-207bbdae5595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '324faeccb7fa41c093bfae329fd960d0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:29.449401 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-80a9dd25-eae9-464e-8fad-f657a6f95d70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:29.459906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 886/1761] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:28 2022] DELETE /identity/v3/projects/30121138fee54489bdebebd5345abac5 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:29.464669 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9dd25-eae9-464e-8fad-f657a6f95d70 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:29.469349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-80a9dd25-eae9-464e-8fad-f657a6f95d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:29.553290 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6ba22e79-2101-4abf-b982-207bbdae5595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '324faeccb7fa41c093bfae329fd960d0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:29.554964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6ba22e79-2101-4abf-b982-207bbdae5595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '324faeccb7fa41c093bfae329fd960d0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:29.577644 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6ba22e79-2101-4abf-b982-207bbdae5595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '324faeccb7fa41c093bfae329fd960d0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:29.579364 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6ba22e79-2101-4abf-b982-207bbdae5595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '324faeccb7fa41c093bfae329fd960d0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:29.601179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 876/1762] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:28 2022] DELETE /identity/v3/users/324faeccb7fa41c093bfae329fd960d0 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:29.606460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563884e-d51a-4d00-87c7-d166378bb480 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:29.607289 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563884e-d51a-4d00-87c7-d166378bb480 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:29.607439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563884e-d51a-4d00-87c7-d166378bb480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:29.672403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-f563884e-d51a-4d00-87c7-d166378bb480 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:29.684961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f563884e-d51a-4d00-87c7-d166378bb480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:29.758146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9dd25-eae9-464e-8fad-f657a6f95d70 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:29.897618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 877/1763] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:29.915098 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:29.938769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563884e-d51a-4d00-87c7-d166378bb480 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:29.941356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f563884e-d51a-4d00-87c7-d166378bb480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:29.947961 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cc107-2d47-4864-a844-4c747675f0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:29.959257 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cc107-2d47-4864-a844-4c747675f0fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:29.959257 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cc107-2d47-4864-a844-4c747675f0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:29.959847 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cc107-2d47-4864-a844-4c747675f0fe admin admin] PATH_INFO: `/v3/users/fd3c2ae863ad4c54aeb2c652d6f7e90f` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:29.963930 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cc107-2d47-4864-a844-4c747675f0fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd3c2ae863ad4c54aeb2c652d6f7e90f)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:29.969079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cc107-2d47-4864-a844-4c747675f0fe admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:29.996051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:30.128259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563884e-d51a-4d00-87c7-d166378bb480 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:30.134729 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2c61d-8604-43d0-ab93-6d975bb63805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:30.136200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2c61d-8604-43d0-ab93-6d975bb63805 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:30.136783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2c61d-8604-43d0-ab93-6d975bb63805 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:30.137269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2c61d-8604-43d0-ab93-6d975bb63805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:30.162616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2c61d-8604-43d0-ab93-6d975bb63805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:30.164805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2c61d-8604-43d0-ab93-6d975bb63805 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:30.181263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 878/1764] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:25:30.340344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-8d2cc107-2d47-4864-a844-4c747675f0fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd3c2ae863ad4c54aeb2c652d6f7e90f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:30.366340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-8d2cc107-2d47-4864-a844-4c747675f0fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd3c2ae863ad4c54aeb2c652d6f7e90f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:30.457599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-8d2cc107-2d47-4864-a844-4c747675f0fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd3c2ae863ad4c54aeb2c652d6f7e90f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:30.469877 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-8d2cc107-2d47-4864-a844-4c747675f0fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd3c2ae863ad4c54aeb2c652d6f7e90f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:30.471902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-8d2cc107-2d47-4864-a844-4c747675f0fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd3c2ae863ad4c54aeb2c652d6f7e90f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:30.484108 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 887/1765] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:29 2022] DELETE /identity/v3/users/fd3c2ae863ad4c54aeb2c652d6f7e90f => generated 0 bytes in 1020 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:30.540520 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1c328-63f9-4eb2-8d99-462b2bd66346 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:30.541383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1c328-63f9-4eb2-8d99-462b2bd66346 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:30.541874 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1c328-63f9-4eb2-8d99-462b2bd66346 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:30.544378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 879/1766] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 31 19:25:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:25:30.567835 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad17a8-b613-4ac4-a37b-ad2a60072602 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:30.568277 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad17a8-b613-4ac4-a37b-ad2a60072602 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:30.568822 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad17a8-b613-4ac4-a37b-ad2a60072602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:30.647981 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-e6ad17a8-b613-4ac4-a37b-ad2a60072602 None None] MFA Rules not processed for user `d461422ce98442bcbe6b254a8dad5e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:30.649439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e6ad17a8-b613-4ac4-a37b-ad2a60072602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:30.772736 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 888/1767] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 31 19:25:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:30.944840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1c328-63f9-4eb2-8d99-462b2bd66346 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:30.949529 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-18a1c328-63f9-4eb2-8d99-462b2bd66346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:30.984425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad17a8-b613-4ac4-a37b-ad2a60072602 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:30.987253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1c328-63f9-4eb2-8d99-462b2bd66346 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:30.988157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e6ad17a8-b613-4ac4-a37b-ad2a60072602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:31.018488 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1202c0c-b30f-4a2f-92ec-6c3edf8eb911 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:31.029232 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1202c0c-b30f-4a2f-92ec-6c3edf8eb911 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:31.029232 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1202c0c-b30f-4a2f-92ec-6c3edf8eb911 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:31.029232 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1202c0c-b30f-4a2f-92ec-6c3edf8eb911 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:31.057155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1202c0c-b30f-4a2f-92ec-6c3edf8eb911 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:31.060924 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1202c0c-b30f-4a2f-92ec-6c3edf8eb911 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:31.111601 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 880/1768] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:31.118515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad17a8-b613-4ac4-a37b-ad2a60072602 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:31.125358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e68ac-1dd4-415c-b68a-514db4183ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:31.125358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e68ac-1dd4-415c-b68a-514db4183ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:31.125622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e68ac-1dd4-415c-b68a-514db4183ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:31.125874 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e68ac-1dd4-415c-b68a-514db4183ae2 admin admin] PATH_INFO: `/v3/projects/85e9ac8822664660b78688c2d8125ea8` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:31.127200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e68ac-1dd4-415c-b68a-514db4183ae2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85e9ac8822664660b78688c2d8125ea8)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:31.129429 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e68ac-1dd4-415c-b68a-514db4183ae2 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:31.316489 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-172e68ac-1dd4-415c-b68a-514db4183ae2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85e9ac8822664660b78688c2d8125ea8', 'request_id': 'req-172e68ac-1dd4-415c-b68a-514db4183ae2'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:31.323641 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-172e68ac-1dd4-415c-b68a-514db4183ae2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85e9ac8822664660b78688c2d8125ea8', 'request_id': 'req-172e68ac-1dd4-415c-b68a-514db4183ae2'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:31.398569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-172e68ac-1dd4-415c-b68a-514db4183ae2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85e9ac8822664660b78688c2d8125ea8', 'request_id': 'req-172e68ac-1dd4-415c-b68a-514db4183ae2'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:31.414639 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-172e68ac-1dd4-415c-b68a-514db4183ae2 admin admin] The token cache is being invalidate because project 85e9ac8822664660b78688c2d8125ea8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:31.414966 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-172e68ac-1dd4-415c-b68a-514db4183ae2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:31.415678 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-172e68ac-1dd4-415c-b68a-514db4183ae2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:31.417403 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 889/1769] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:30 2022] DELETE /identity/v3/projects/85e9ac8822664660b78688c2d8125ea8 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:31.421749 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1202c0c-b30f-4a2f-92ec-6c3edf8eb911 service nova] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:31.440119 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e68ac-1dd4-415c-b68a-514db4183ae2 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:31.441843 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-172e68ac-1dd4-415c-b68a-514db4183ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:31.443915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1202c0c-b30f-4a2f-92ec-6c3edf8eb911 service nova] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:31.585761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd92e95-2a96-4a57-855d-38ce357fad93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:31.591790 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd92e95-2a96-4a57-855d-38ce357fad93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:31.591790 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd92e95-2a96-4a57-855d-38ce357fad93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:31.591790 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd92e95-2a96-4a57-855d-38ce357fad93 admin admin] PATH_INFO: `/v3/users/499f5a13a2394570982138745db67308` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:31.591790 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd92e95-2a96-4a57-855d-38ce357fad93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=499f5a13a2394570982138745db67308)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:31.591790 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd92e95-2a96-4a57-855d-38ce357fad93 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:31.646171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e68ac-1dd4-415c-b68a-514db4183ae2 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:31.665472 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2fb54-0b52-4d48-ab59-0a1cb59ae45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:31.666454 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2fb54-0b52-4d48-ab59-0a1cb59ae45f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:31.666808 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2fb54-0b52-4d48-ab59-0a1cb59ae45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:31.667210 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2fb54-0b52-4d48-ab59-0a1cb59ae45f admin admin] PATH_INFO: `/v3/projects/43bd55a6e88b4038a0772445761e7135` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:31.668755 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2fb54-0b52-4d48-ab59-0a1cb59ae45f admin admin] RBAC: Authorizing `identity:delete_project(project_id=43bd55a6e88b4038a0772445761e7135)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:31.681294 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2fb54-0b52-4d48-ab59-0a1cb59ae45f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:31.926030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-5fd92e95-2a96-4a57-855d-38ce357fad93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '499f5a13a2394570982138745db67308'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:31.927704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-78c2fb54-0b52-4d48-ab59-0a1cb59ae45f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43bd55a6e88b4038a0772445761e7135', 'request_id': 'req-78c2fb54-0b52-4d48-ab59-0a1cb59ae45f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:31.955868 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-78c2fb54-0b52-4d48-ab59-0a1cb59ae45f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43bd55a6e88b4038a0772445761e7135', 'request_id': 'req-78c2fb54-0b52-4d48-ab59-0a1cb59ae45f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:32.010337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-5fd92e95-2a96-4a57-855d-38ce357fad93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '499f5a13a2394570982138745db67308'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:32.011100 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-5fd92e95-2a96-4a57-855d-38ce357fad93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '499f5a13a2394570982138745db67308'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:32.043514 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-5fd92e95-2a96-4a57-855d-38ce357fad93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '499f5a13a2394570982138745db67308'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:32.045633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-5fd92e95-2a96-4a57-855d-38ce357fad93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '499f5a13a2394570982138745db67308'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:32.088630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-78c2fb54-0b52-4d48-ab59-0a1cb59ae45f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43bd55a6e88b4038a0772445761e7135', 'request_id': 'req-78c2fb54-0b52-4d48-ab59-0a1cb59ae45f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:32.088925 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 881/1770] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:31 2022] DELETE /identity/v3/users/499f5a13a2394570982138745db67308 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:32.138636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-78c2fb54-0b52-4d48-ab59-0a1cb59ae45f admin admin] The token cache is being invalidate because project 43bd55a6e88b4038a0772445761e7135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:32.139123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-78c2fb54-0b52-4d48-ab59-0a1cb59ae45f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:32.139874 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-78c2fb54-0b52-4d48-ab59-0a1cb59ae45f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:32.141676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 890/1771] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:31 2022] DELETE /identity/v3/projects/43bd55a6e88b4038a0772445761e7135 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:32.147272 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20266ad6-5c86-4d66-bf9c-7d4d31eef25b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:32.148208 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20266ad6-5c86-4d66-bf9c-7d4d31eef25b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:32.148671 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20266ad6-5c86-4d66-bf9c-7d4d31eef25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:32.194428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-20266ad6-5c86-4d66-bf9c-7d4d31eef25b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:32.196082 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-20266ad6-5c86-4d66-bf9c-7d4d31eef25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:32.358910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 882/1772] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:32.364615 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf214d0-fd69-4906-8783-64b72b19c257 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:32.365098 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf214d0-fd69-4906-8783-64b72b19c257 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:32.365566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf214d0-fd69-4906-8783-64b72b19c257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:32.420606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-4bf214d0-fd69-4906-8783-64b72b19c257 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:32.421946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4bf214d0-fd69-4906-8783-64b72b19c257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:32.431414 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 891/1773] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:32.435669 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20266ad6-5c86-4d66-bf9c-7d4d31eef25b None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:32.456945 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf214d0-fd69-4906-8783-64b72b19c257 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:32.459091 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4bf214d0-fd69-4906-8783-64b72b19c257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:32.467096 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20266ad6-5c86-4d66-bf9c-7d4d31eef25b None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:32.477241 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf214d0-fd69-4906-8783-64b72b19c257 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:32.478626 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26221f5-be62-4156-823f-17cda85982eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:32.479461 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26221f5-be62-4156-823f-17cda85982eb admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:32.479988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26221f5-be62-4156-823f-17cda85982eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:32.481179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26221f5-be62-4156-823f-17cda85982eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:32.481179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26221f5-be62-4156-823f-17cda85982eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:25:32.488698 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26221f5-be62-4156-823f-17cda85982eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:32.490360 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26221f5-be62-4156-823f-17cda85982eb admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:32.490819 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8081d6-be78-4f1f-b960-faf9e4ab126f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:32.491703 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8081d6-be78-4f1f-b960-faf9e4ab126f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:32.493101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8081d6-be78-4f1f-b960-faf9e4ab126f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:32.493574 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8081d6-be78-4f1f-b960-faf9e4ab126f admin admin] PATH_INFO: `/v3/projects/1eb0d93260b44c6ea812bcc790e24dab` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:32.497424 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8081d6-be78-4f1f-b960-faf9e4ab126f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eb0d93260b44c6ea812bcc790e24dab)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:32.523041 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8081d6-be78-4f1f-b960-faf9e4ab126f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:32.541148 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 883/1774] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:25:32.545986 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26221f5-be62-4156-823f-17cda85982eb admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:32.580466 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26221f5-be62-4156-823f-17cda85982eb admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:32.600551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d709a4d-c060-45e6-9a93-48a548cf0b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:32.602443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d709a4d-c060-45e6-9a93-48a548cf0b59 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:32.603888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d709a4d-c060-45e6-9a93-48a548cf0b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:32.604342 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d709a4d-c060-45e6-9a93-48a548cf0b59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:32.605219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d709a4d-c060-45e6-9a93-48a548cf0b59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:32.609716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d709a4d-c060-45e6-9a93-48a548cf0b59 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:32.728329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-bd8081d6-be78-4f1f-b960-faf9e4ab126f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eb0d93260b44c6ea812bcc790e24dab', 'request_id': 'req-bd8081d6-be78-4f1f-b960-faf9e4ab126f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:32.735366 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-bd8081d6-be78-4f1f-b960-faf9e4ab126f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eb0d93260b44c6ea812bcc790e24dab', 'request_id': 'req-bd8081d6-be78-4f1f-b960-faf9e4ab126f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:32.765192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 884/1775] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:32 2022] POST /identity/v3/projects => generated 377 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:32.766500 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d709a4d-c060-45e6-9a93-48a548cf0b59 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:32.771048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:32.804417 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-bd8081d6-be78-4f1f-b960-faf9e4ab126f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eb0d93260b44c6ea812bcc790e24dab', 'request_id': 'req-bd8081d6-be78-4f1f-b960-faf9e4ab126f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:32.819705 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-bd8081d6-be78-4f1f-b960-faf9e4ab126f admin admin] The token cache is being invalidate because project 1eb0d93260b44c6ea812bcc790e24dab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:32.824995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-bd8081d6-be78-4f1f-b960-faf9e4ab126f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:32.827856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-bd8081d6-be78-4f1f-b960-faf9e4ab126f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:32.831663 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 892/1776] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:32 2022] DELETE /identity/v3/projects/1eb0d93260b44c6ea812bcc790e24dab => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:32.835828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8081d6-be78-4f1f-b960-faf9e4ab126f admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:32.837709 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-bd8081d6-be78-4f1f-b960-faf9e4ab126f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:32.865056 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:32.921759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d709a4d-c060-45e6-9a93-48a548cf0b59 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:32.928008 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7ddba-135b-4952-b543-a5203f75f321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:32.928974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7ddba-135b-4952-b543-a5203f75f321 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:32.929247 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7ddba-135b-4952-b543-a5203f75f321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:32.929519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7ddba-135b-4952-b543-a5203f75f321 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:32.930578 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7ddba-135b-4952-b543-a5203f75f321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:32.932880 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7ddba-135b-4952-b543-a5203f75f321 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:33.075272 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8081d6-be78-4f1f-b960-faf9e4ab126f admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:33.094134 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea214f5-5ca1-4aeb-a27f-7f55b833c214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:33.094806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea214f5-5ca1-4aeb-a27f-7f55b833c214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:33.095117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea214f5-5ca1-4aeb-a27f-7f55b833c214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:33.105632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea214f5-5ca1-4aeb-a27f-7f55b833c214 admin admin] PATH_INFO: `/v3/users/6b343dd958204fce9bd43f0809bd0e9f` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:33.105880 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 885/1777] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:32 2022] POST /identity/v3/users => generated 289 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:33.105959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea214f5-5ca1-4aeb-a27f-7f55b833c214 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b343dd958204fce9bd43f0809bd0e9f)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:33.106029 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7ddba-135b-4952-b543-a5203f75f321 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:33.106097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea214f5-5ca1-4aeb-a27f-7f55b833c214 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:33.106175 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d8c7ddba-135b-4952-b543-a5203f75f321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:33.130994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7ddba-135b-4952-b543-a5203f75f321 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:33.139710 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c579e-bfc5-4376-8cb3-51403a0733a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:33.142169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c579e-bfc5-4376-8cb3-51403a0733a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:33.142169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c579e-bfc5-4376-8cb3-51403a0733a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:33.142169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c579e-bfc5-4376-8cb3-51403a0733a1 admin admin] PATH_INFO: `/v3/projects/14d845bf302445b89c3048e4aeebba30` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:33.142169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c579e-bfc5-4376-8cb3-51403a0733a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14d845bf302445b89c3048e4aeebba30)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:33.148853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c579e-bfc5-4376-8cb3-51403a0733a1 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:33.431520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2ea214f5-5ca1-4aeb-a27f-7f55b833c214 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b343dd958204fce9bd43f0809bd0e9f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:33.432139 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-455c579e-bfc5-4376-8cb3-51403a0733a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14d845bf302445b89c3048e4aeebba30', 'request_id': 'req-455c579e-bfc5-4376-8cb3-51403a0733a1'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:33.443801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-455c579e-bfc5-4376-8cb3-51403a0733a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14d845bf302445b89c3048e4aeebba30', 'request_id': 'req-455c579e-bfc5-4376-8cb3-51403a0733a1'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:33.452292 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-455c579e-bfc5-4376-8cb3-51403a0733a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14d845bf302445b89c3048e4aeebba30', 'request_id': 'req-455c579e-bfc5-4376-8cb3-51403a0733a1'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:33.476841 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2ea214f5-5ca1-4aeb-a27f-7f55b833c214 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b343dd958204fce9bd43f0809bd0e9f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:33.570171 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-455c579e-bfc5-4376-8cb3-51403a0733a1 admin admin] The token cache is being invalidate because project 14d845bf302445b89c3048e4aeebba30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:33.570660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-455c579e-bfc5-4376-8cb3-51403a0733a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:33.572025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-455c579e-bfc5-4376-8cb3-51403a0733a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:33.582504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 886/1778] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:33 2022] DELETE /identity/v3/projects/14d845bf302445b89c3048e4aeebba30 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:33.582504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c579e-bfc5-4376-8cb3-51403a0733a1 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:33.582504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-455c579e-bfc5-4376-8cb3-51403a0733a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:33.622691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2ea214f5-5ca1-4aeb-a27f-7f55b833c214 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b343dd958204fce9bd43f0809bd0e9f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:33.633463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2ea214f5-5ca1-4aeb-a27f-7f55b833c214 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b343dd958204fce9bd43f0809bd0e9f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:33.636258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2ea214f5-5ca1-4aeb-a27f-7f55b833c214 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b343dd958204fce9bd43f0809bd0e9f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:33.652747 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 893/1779] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:32 2022] DELETE /identity/v3/users/6b343dd958204fce9bd43f0809bd0e9f => generated 0 bytes in 817 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:33.654965 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2e3c1-dd71-4815-9861-695768fc68e9 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:33.660887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2e3c1-dd71-4815-9861-695768fc68e9 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:33.660887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2e3c1-dd71-4815-9861-695768fc68e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:33.705273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-9fa2e3c1-dd71-4815-9861-695768fc68e9 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:33.707455 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9fa2e3c1-dd71-4815-9861-695768fc68e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:33.767310 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c579e-bfc5-4376-8cb3-51403a0733a1 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:33.888135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 894/1780] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:33.917319 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15845fcc-002f-40f7-af12-9d30df4176aa None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:33.917910 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15845fcc-002f-40f7-af12-9d30df4176aa None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:33.917910 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15845fcc-002f-40f7-af12-9d30df4176aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:33.965587 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d5edb-61d5-4f24-bfad-f8d5dfda610e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:33.967997 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d5edb-61d5-4f24-bfad-f8d5dfda610e admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:33.983447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d5edb-61d5-4f24-bfad-f8d5dfda610e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:33.983447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d5edb-61d5-4f24-bfad-f8d5dfda610e admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:33.983447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d5edb-61d5-4f24-bfad-f8d5dfda610e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:33.983827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-15845fcc-002f-40f7-af12-9d30df4176aa None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:33.983931 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d5edb-61d5-4f24-bfad-f8d5dfda610e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:33.984017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-15845fcc-002f-40f7-af12-9d30df4176aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:34.009583 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 895/1781] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:34.010945 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15845fcc-002f-40f7-af12-9d30df4176aa None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:34.028124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15845fcc-002f-40f7-af12-9d30df4176aa None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:34.034887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d371b4-5ce1-4642-b31e-4d5af36a95ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:34.035458 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 887/1782] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:34.035730 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d371b4-5ce1-4642-b31e-4d5af36a95ff admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:34.036019 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d371b4-5ce1-4642-b31e-4d5af36a95ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:34.036300 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d371b4-5ce1-4642-b31e-4d5af36a95ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:34.037048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d371b4-5ce1-4642-b31e-4d5af36a95ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:25:34.037332 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d371b4-5ce1-4642-b31e-4d5af36a95ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:34.039025 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d371b4-5ce1-4642-b31e-4d5af36a95ff admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:34.039777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d5edb-61d5-4f24-bfad-f8d5dfda610e admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:34.043155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-528d5edb-61d5-4f24-bfad-f8d5dfda610e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:34.059233 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 896/1783] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:25:34.060985 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d371b4-5ce1-4642-b31e-4d5af36a95ff admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:34.063140 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d5edb-61d5-4f24-bfad-f8d5dfda610e admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:34.079979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d371b4-5ce1-4642-b31e-4d5af36a95ff admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:34.081945 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76551964-f5e6-49ad-b8c4-4a2e04ba7523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:34.082682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76551964-f5e6-49ad-b8c4-4a2e04ba7523 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:34.083089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76551964-f5e6-49ad-b8c4-4a2e04ba7523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:34.083460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76551964-f5e6-49ad-b8c4-4a2e04ba7523 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:34.084214 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76551964-f5e6-49ad-b8c4-4a2e04ba7523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:34.085929 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76551964-f5e6-49ad-b8c4-4a2e04ba7523 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:34.086156 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121b5fb-27e2-4ea3-8fb0-7159c4f8b75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:34.086950 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121b5fb-27e2-4ea3-8fb0-7159c4f8b75e admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:34.087321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121b5fb-27e2-4ea3-8fb0-7159c4f8b75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:34.087637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121b5fb-27e2-4ea3-8fb0-7159c4f8b75e admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:34.088515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121b5fb-27e2-4ea3-8fb0-7159c4f8b75e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:34.090570 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121b5fb-27e2-4ea3-8fb0-7159c4f8b75e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:34.133311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 888/1784] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:34.144387 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76551964-f5e6-49ad-b8c4-4a2e04ba7523 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:34.156841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76551964-f5e6-49ad-b8c4-4a2e04ba7523 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:34.162729 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450cd7b-9193-4d25-8bf3-d6f313708ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:34.163717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450cd7b-9193-4d25-8bf3-d6f313708ef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:34.165361 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450cd7b-9193-4d25-8bf3-d6f313708ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:34.165361 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450cd7b-9193-4d25-8bf3-d6f313708ef9 admin admin] PATH_INFO: `/v3/projects/b9dca28a00cc42c3bdf64febc9089fb2/users/b5d01942fa8b48b19cee2003966be417/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:34.193989 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450cd7b-9193-4d25-8bf3-d6f313708ef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9dca28a00cc42c3bdf64febc9089fb2, user_id=b5d01942fa8b48b19cee2003966be417, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:34.197718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450cd7b-9193-4d25-8bf3-d6f313708ef9 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:34.201378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 897/1785] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:34 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:34.202538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121b5fb-27e2-4ea3-8fb0-7159c4f8b75e admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:34.302598 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 889/1786] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:34 2022] PUT /identity/v3/projects/b9dca28a00cc42c3bdf64febc9089fb2/users/b5d01942fa8b48b19cee2003966be417/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:34.315881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7c6e6-9a31-4309-a1fd-52b56b11c9c7 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:34.316566 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7c6e6-9a31-4309-a1fd-52b56b11c9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:34.317346 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7c6e6-9a31-4309-a1fd-52b56b11c9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:34.348390 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121b5fb-27e2-4ea3-8fb0-7159c4f8b75e admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:34.388694 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-acf7c6e6-9a31-4309-a1fd-52b56b11c9c7 None None] MFA Rules not processed for user `b5d01942fa8b48b19cee2003966be417`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:34.391435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-acf7c6e6-9a31-4309-a1fd-52b56b11c9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:34.516568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9227e719-bc37-49ab-8f02-c1842116096e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:34.518229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227e719-bc37-49ab-8f02-c1842116096e admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:34.518526 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227e719-bc37-49ab-8f02-c1842116096e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:34.518855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227e719-bc37-49ab-8f02-c1842116096e admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:34.520283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9227e719-bc37-49ab-8f02-c1842116096e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:34.526125 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9227e719-bc37-49ab-8f02-c1842116096e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:34.533920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 890/1787] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:34.538017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7c6e6-9a31-4309-a1fd-52b56b11c9c7 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:34.585131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7c6e6-9a31-4309-a1fd-52b56b11c9c7 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:34.592365 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef3ede-b58c-48ff-a410-8092066f1299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:34.594175 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef3ede-b58c-48ff-a410-8092066f1299 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:34.594614 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef3ede-b58c-48ff-a410-8092066f1299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:34.595110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef3ede-b58c-48ff-a410-8092066f1299 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:34.595908 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef3ede-b58c-48ff-a410-8092066f1299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:34.599141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef3ede-b58c-48ff-a410-8092066f1299 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:34.695088 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 898/1788] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:34 2022] POST /identity/v3/users => generated 289 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:34.696795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9227e719-bc37-49ab-8f02-c1842116096e admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:34.721825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 891/1789] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:34 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:25:34.722066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9227e719-bc37-49ab-8f02-c1842116096e admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:34.724692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef3ede-b58c-48ff-a410-8092066f1299 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:34.887696 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14e6a1-d31e-4cc6-8edb-e27ee0476e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:34.889169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14e6a1-d31e-4cc6-8edb-e27ee0476e09 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:34.889560 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14e6a1-d31e-4cc6-8edb-e27ee0476e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:34.889845 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14e6a1-d31e-4cc6-8edb-e27ee0476e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:34.890512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14e6a1-d31e-4cc6-8edb-e27ee0476e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:34.891235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef3ede-b58c-48ff-a410-8092066f1299 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:34.896906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14e6a1-d31e-4cc6-8edb-e27ee0476e09 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:34.918334 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22c0d5-9c50-4104-81c0-aef71b4e697e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:34.918940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22c0d5-9c50-4104-81c0-aef71b4e697e admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:34.919346 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22c0d5-9c50-4104-81c0-aef71b4e697e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:34.919784 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22c0d5-9c50-4104-81c0-aef71b4e697e admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:34.920576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22c0d5-9c50-4104-81c0-aef71b4e697e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:34.922499 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22c0d5-9c50-4104-81c0-aef71b4e697e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:34.960133 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 899/1790] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:34.963546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14e6a1-d31e-4cc6-8edb-e27ee0476e09 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:34.998561 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14e6a1-d31e-4cc6-8edb-e27ee0476e09 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:35.004206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c78e0-507b-4bc6-9d62-721dc5cd4d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:35.004951 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c78e0-507b-4bc6-9d62-721dc5cd4d6f admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:35.005288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c78e0-507b-4bc6-9d62-721dc5cd4d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:35.005805 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c78e0-507b-4bc6-9d62-721dc5cd4d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:35.006699 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c78e0-507b-4bc6-9d62-721dc5cd4d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:35.008621 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c78e0-507b-4bc6-9d62-721dc5cd4d6f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:35.045804 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 900/1791] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:35.050002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c78e0-507b-4bc6-9d62-721dc5cd4d6f admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:35.071634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 892/1792] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:34 2022] POST /identity/v3/users => generated 295 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:25:35.078195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c78e0-507b-4bc6-9d62-721dc5cd4d6f admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:35.080717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22c0d5-9c50-4104-81c0-aef71b4e697e admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:35.086623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf765fa-4d13-4961-bdf8-f1167b87d6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:35.087483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf765fa-4d13-4961-bdf8-f1167b87d6dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:35.088017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf765fa-4d13-4961-bdf8-f1167b87d6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:35.088298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf765fa-4d13-4961-bdf8-f1167b87d6dd admin admin] PATH_INFO: `/v3/projects/b0044bfd07484ff29152de409375ac37/users/e5e7d2c63a4b48e59ffeb9f153a214c5/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:35.091965 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22c0d5-9c50-4104-81c0-aef71b4e697e admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:35.097431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd213d62-abee-4ab6-a5f8-f4c6d5801d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:35.098223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd213d62-abee-4ab6-a5f8-f4c6d5801d65 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:35.098594 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd213d62-abee-4ab6-a5f8-f4c6d5801d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:35.099011 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd213d62-abee-4ab6-a5f8-f4c6d5801d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:35.099770 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd213d62-abee-4ab6-a5f8-f4c6d5801d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:35.102071 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd213d62-abee-4ab6-a5f8-f4c6d5801d65 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:35.116986 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 893/1793] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:25:35.124031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd213d62-abee-4ab6-a5f8-f4c6d5801d65 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:35.138789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf765fa-4d13-4961-bdf8-f1167b87d6dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0044bfd07484ff29152de409375ac37, user_id=e5e7d2c63a4b48e59ffeb9f153a214c5, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:35.149931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf765fa-4d13-4961-bdf8-f1167b87d6dd admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:35.150161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd213d62-abee-4ab6-a5f8-f4c6d5801d65 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:35.160115 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830e9d9-14b6-4a23-9ff0-e15068201053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:35.161099 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830e9d9-14b6-4a23-9ff0-e15068201053 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:35.166588 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830e9d9-14b6-4a23-9ff0-e15068201053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:35.166588 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830e9d9-14b6-4a23-9ff0-e15068201053 admin admin] PATH_INFO: `/v3/projects/57beea4099084ca89630de811752d15b/users/b1628ca0fb584c6290fa0dfe1cea9136/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:35.198020 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830e9d9-14b6-4a23-9ff0-e15068201053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57beea4099084ca89630de811752d15b, user_id=b1628ca0fb584c6290fa0dfe1cea9136, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:35.201993 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830e9d9-14b6-4a23-9ff0-e15068201053 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:35.245370 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 901/1794] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:35 2022] PUT /identity/v3/projects/b0044bfd07484ff29152de409375ac37/users/e5e7d2c63a4b48e59ffeb9f153a214c5/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:35.261665 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0fea5-5f3c-4b3b-b922-e8bbc7a8625c None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:35.262366 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0fea5-5f3c-4b3b-b922-e8bbc7a8625c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:35.279199 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0fea5-5f3c-4b3b-b922-e8bbc7a8625c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:35.335403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 894/1795] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:35 2022] PUT /identity/v3/projects/57beea4099084ca89630de811752d15b/users/b1628ca0fb584c6290fa0dfe1cea9136/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:35.336744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830e9d9-14b6-4a23-9ff0-e15068201053 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:35.359808 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-66f0fea5-5f3c-4b3b-b922-e8bbc7a8625c None None] MFA Rules not processed for user `e5e7d2c63a4b48e59ffeb9f153a214c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:35.359808 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-66f0fea5-5f3c-4b3b-b922-e8bbc7a8625c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:35.428712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:35.500479 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830e9d9-14b6-4a23-9ff0-e15068201053 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:35.506640 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287df9f-1f54-475d-ac3f-aad370cff7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:35.507319 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287df9f-1f54-475d-ac3f-aad370cff7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:35.507650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287df9f-1f54-475d-ac3f-aad370cff7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:35.507933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287df9f-1f54-475d-ac3f-aad370cff7a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:35.508582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287df9f-1f54-475d-ac3f-aad370cff7a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:35.510200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287df9f-1f54-475d-ac3f-aad370cff7a2 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:35.526722 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 895/1796] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:35.536586 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:35.539007 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287df9f-1f54-475d-ac3f-aad370cff7a2 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:35.552396 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287df9f-1f54-475d-ac3f-aad370cff7a2 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:35.558631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c4808-f799-4026-8bb0-f92cef1431dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:35.563664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c4808-f799-4026-8bb0-f92cef1431dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:35.563664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c4808-f799-4026-8bb0-f92cef1431dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:35.563664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c4808-f799-4026-8bb0-f92cef1431dc admin admin] PATH_INFO: `/v3/projects/57beea4099084ca89630de811752d15b/users/b1628ca0fb584c6290fa0dfe1cea9136/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:35.565342 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c4808-f799-4026-8bb0-f92cef1431dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=57beea4099084ca89630de811752d15b, user_id=b1628ca0fb584c6290fa0dfe1cea9136, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:35.572224 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c4808-f799-4026-8bb0-f92cef1431dc admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:35.589155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 896/1797] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:35 2022] PUT /identity/v3/projects/57beea4099084ca89630de811752d15b/users/b1628ca0fb584c6290fa0dfe1cea9136/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:35.590425 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c4808-f799-4026-8bb0-f92cef1431dc admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:35.600432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8b3c4808-f799-4026-8bb0-f92cef1431dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:35.627616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 902/1798] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:35.631918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bcee3-5ab2-4a0c-bac3-5e3a8cc778ec None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:35.631918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bcee3-5ab2-4a0c-bac3-5e3a8cc778ec None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:35.631918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bcee3-5ab2-4a0c-bac3-5e3a8cc778ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:35.715418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-6a8bcee3-5ab2-4a0c-bac3-5e3a8cc778ec None None] MFA Rules not processed for user `b1628ca0fb584c6290fa0dfe1cea9136`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:35.726288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6a8bcee3-5ab2-4a0c-bac3-5e3a8cc778ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:35.732817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c4808-f799-4026-8bb0-f92cef1431dc admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:35.742595 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529990de-1d92-419e-acfb-7d93b44b3e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:35.744637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529990de-1d92-419e-acfb-7d93b44b3e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:35.745054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529990de-1d92-419e-acfb-7d93b44b3e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:35.745703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529990de-1d92-419e-acfb-7d93b44b3e9a admin admin] PATH_INFO: `/v3/projects/cc78554a7d494875bde5f29ff8b9b634` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:35.747858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529990de-1d92-419e-acfb-7d93b44b3e9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc78554a7d494875bde5f29ff8b9b634)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:35.752202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529990de-1d92-419e-acfb-7d93b44b3e9a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:35.836852 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 903/1799] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:35.838011 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bcee3-5ab2-4a0c-bac3-5e3a8cc778ec None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:35.841142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6a8bcee3-5ab2-4a0c-bac3-5e3a8cc778ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:35.957592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bcee3-5ab2-4a0c-bac3-5e3a8cc778ec None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:35.962184 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a87767-104a-47aa-973a-74267625685c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:35.963106 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a87767-104a-47aa-973a-74267625685c service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:35.963454 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a87767-104a-47aa-973a-74267625685c service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:35.963781 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a87767-104a-47aa-973a-74267625685c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:36.003487 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-529990de-1d92-419e-acfb-7d93b44b3e9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc78554a7d494875bde5f29ff8b9b634', 'request_id': 'req-529990de-1d92-419e-acfb-7d93b44b3e9a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:36.013777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-529990de-1d92-419e-acfb-7d93b44b3e9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc78554a7d494875bde5f29ff8b9b634', 'request_id': 'req-529990de-1d92-419e-acfb-7d93b44b3e9a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:36.023970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-529990de-1d92-419e-acfb-7d93b44b3e9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc78554a7d494875bde5f29ff8b9b634', 'request_id': 'req-529990de-1d92-419e-acfb-7d93b44b3e9a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:36.049994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a87767-104a-47aa-973a-74267625685c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:36.051630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a87767-104a-47aa-973a-74267625685c service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:36.069652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 904/1800] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:36.070494 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a87767-104a-47aa-973a-74267625685c service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:36.072253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a1a87767-104a-47aa-973a-74267625685c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:36.093240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a87767-104a-47aa-973a-74267625685c service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:36.175284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c583c3-d0c6-478b-8109-7a0df9665455 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:36.175284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c583c3-d0c6-478b-8109-7a0df9665455 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:36.175284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c583c3-d0c6-478b-8109-7a0df9665455 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:36.175284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c583c3-d0c6-478b-8109-7a0df9665455 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:36.184546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-529990de-1d92-419e-acfb-7d93b44b3e9a admin admin] The token cache is being invalidate because project cc78554a7d494875bde5f29ff8b9b634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:36.185853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-529990de-1d92-419e-acfb-7d93b44b3e9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:36.187316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-529990de-1d92-419e-acfb-7d93b44b3e9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:36.189380 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 897/1801] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:35 2022] DELETE /identity/v3/projects/cc78554a7d494875bde5f29ff8b9b634 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:36.194736 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd95a8b-ab48-4c9e-a110-a430f2be27d6 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:36.195149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd95a8b-ab48-4c9e-a110-a430f2be27d6 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:36.195636 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd95a8b-ab48-4c9e-a110-a430f2be27d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:36.245288 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-6bd95a8b-ab48-4c9e-a110-a430f2be27d6 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:36.246865 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-6bd95a8b-ab48-4c9e-a110-a430f2be27d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:36.301603 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c583c3-d0c6-478b-8109-7a0df9665455 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:36.303311 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c583c3-d0c6-478b-8109-7a0df9665455 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:36.315974 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 905/1802] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:36.375969 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 898/1803] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:25:36.383923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362aedd-2e4c-4d98-8727-7eaf6d14cc1d None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:36.388801 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362aedd-2e4c-4d98-8727-7eaf6d14cc1d None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:36.388801 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362aedd-2e4c-4d98-8727-7eaf6d14cc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:36.423761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-7362aedd-2e4c-4d98-8727-7eaf6d14cc1d None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:36.428376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7362aedd-2e4c-4d98-8727-7eaf6d14cc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:36.462910 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 906/1804] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:36.465186 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd95a8b-ab48-4c9e-a110-a430f2be27d6 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:36.475853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd95a8b-ab48-4c9e-a110-a430f2be27d6 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:36.481261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a9955-1135-4150-830f-3515b460b8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:36.482038 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a9955-1135-4150-830f-3515b460b8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:36.482580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a9955-1135-4150-830f-3515b460b8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:36.482918 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a9955-1135-4150-830f-3515b460b8f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:36.483717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a9955-1135-4150-830f-3515b460b8f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:25:36.484028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a9955-1135-4150-830f-3515b460b8f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:36.485645 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a9955-1135-4150-830f-3515b460b8f7 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:36.502718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 899/1805] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:25:36.504001 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362aedd-2e4c-4d98-8727-7eaf6d14cc1d None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:36.515823 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362aedd-2e4c-4d98-8727-7eaf6d14cc1d None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:36.521294 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f104417-8cf3-4802-a8f9-b04e1f3be0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:36.521922 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f104417-8cf3-4802-a8f9-b04e1f3be0fa admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:36.522594 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f104417-8cf3-4802-a8f9-b04e1f3be0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:36.522899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f104417-8cf3-4802-a8f9-b04e1f3be0fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:36.523564 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f104417-8cf3-4802-a8f9-b04e1f3be0fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:36.525476 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f104417-8cf3-4802-a8f9-b04e1f3be0fa admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:36.633589 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 907/1806] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:36 2022] POST /identity/v3/projects => generated 367 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:36.635356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a9955-1135-4150-830f-3515b460b8f7 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:36.646612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a9955-1135-4150-830f-3515b460b8f7 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:36.705059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd8ed8-942b-49e4-9402-503681c42acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:36.705671 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd8ed8-942b-49e4-9402-503681c42acc admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:36.705950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd8ed8-942b-49e4-9402-503681c42acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:36.706227 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd8ed8-942b-49e4-9402-503681c42acc admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:36.706853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd8ed8-942b-49e4-9402-503681c42acc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:36.708780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd8ed8-942b-49e4-9402-503681c42acc admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:36.803918 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 900/1807] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:36 2022] POST /identity/v3/users => generated 284 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:36.808094 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f104417-8cf3-4802-a8f9-b04e1f3be0fa admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:36.818965 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f104417-8cf3-4802-a8f9-b04e1f3be0fa admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:36.823554 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6e93e-5e8d-4537-a29c-730cb05573cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:36.824962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6e93e-5e8d-4537-a29c-730cb05573cb admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:36.824962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6e93e-5e8d-4537-a29c-730cb05573cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:36.824962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6e93e-5e8d-4537-a29c-730cb05573cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:36.825268 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6e93e-5e8d-4537-a29c-730cb05573cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:36.827686 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6e93e-5e8d-4537-a29c-730cb05573cb admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:36.837760 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 908/1808] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:36.841694 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd8ed8-942b-49e4-9402-503681c42acc admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:36.853907 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd8ed8-942b-49e4-9402-503681c42acc admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:36.859596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9e269-c804-4c73-8147-da51c5e0ac41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:36.859596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9e269-c804-4c73-8147-da51c5e0ac41 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:36.859596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9e269-c804-4c73-8147-da51c5e0ac41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:36.859917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9e269-c804-4c73-8147-da51c5e0ac41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:36.860517 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9e269-c804-4c73-8147-da51c5e0ac41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:36.862110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9e269-c804-4c73-8147-da51c5e0ac41 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:36.872918 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 901/1809] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:36.877705 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6e93e-5e8d-4537-a29c-730cb05573cb admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:36.887896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6e93e-5e8d-4537-a29c-730cb05573cb admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:36.893415 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c1333-d409-4265-a65e-b7e6fe107546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:36.894131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c1333-d409-4265-a65e-b7e6fe107546 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:36.894418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c1333-d409-4265-a65e-b7e6fe107546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:36.894698 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c1333-d409-4265-a65e-b7e6fe107546 admin admin] PATH_INFO: `/v3/projects/27e387d970974c54a75e48614fcb993f/users/3c3da37755134ae2b1ba82803da56f78/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:36.910043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c1333-d409-4265-a65e-b7e6fe107546 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27e387d970974c54a75e48614fcb993f, user_id=3c3da37755134ae2b1ba82803da56f78, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:36.912775 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c1333-d409-4265-a65e-b7e6fe107546 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:37.037556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 909/1810] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:36 2022] PUT /identity/v3/projects/27e387d970974c54a75e48614fcb993f/users/3c3da37755134ae2b1ba82803da56f78/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:37.050335 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12960b28-7909-4179-b7f8-143f5e7a45bf None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:37.050919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12960b28-7909-4179-b7f8-143f5e7a45bf None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:37.050919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12960b28-7909-4179-b7f8-143f5e7a45bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:37.084537 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-12960b28-7909-4179-b7f8-143f5e7a45bf None None] MFA Rules not processed for user `3c3da37755134ae2b1ba82803da56f78`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:37.085779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-12960b28-7909-4179-b7f8-143f5e7a45bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:37.166703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 902/1811] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:37.171387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c1333-d409-4265-a65e-b7e6fe107546 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:37.245686 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c1333-d409-4265-a65e-b7e6fe107546 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:37.251401 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbc2e5-8960-445e-a5c9-f32aaa6076e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:37.252088 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbc2e5-8960-445e-a5c9-f32aaa6076e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:37.252434 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbc2e5-8960-445e-a5c9-f32aaa6076e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:37.252775 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbc2e5-8960-445e-a5c9-f32aaa6076e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:37.253489 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbc2e5-8960-445e-a5c9-f32aaa6076e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:37.255432 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbc2e5-8960-445e-a5c9-f32aaa6076e7 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:37.382814 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 910/1812] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:37 2022] POST /identity/v3/projects => generated 369 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:25:37.387018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12960b28-7909-4179-b7f8-143f5e7a45bf None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:37.398409 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12960b28-7909-4179-b7f8-143f5e7a45bf None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:37.458944 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c03f23-80a6-453b-a028-73018b29d08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:37.459620 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c03f23-80a6-453b-a028-73018b29d08b admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:37.459910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c03f23-80a6-453b-a028-73018b29d08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:37.460186 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c03f23-80a6-453b-a028-73018b29d08b admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:37.460826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c03f23-80a6-453b-a028-73018b29d08b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:37.462757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c03f23-80a6-453b-a028-73018b29d08b admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:37.576163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 903/1813] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:37 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:25:37.580167 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbc2e5-8960-445e-a5c9-f32aaa6076e7 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:37.590657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbc2e5-8960-445e-a5c9-f32aaa6076e7 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:37.596044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6494c-53e3-4494-ba23-9743e29f2c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:37.596755 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6494c-53e3-4494-ba23-9743e29f2c99 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:37.597090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6494c-53e3-4494-ba23-9743e29f2c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:37.597370 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6494c-53e3-4494-ba23-9743e29f2c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:37.598035 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6494c-53e3-4494-ba23-9743e29f2c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:37.599706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6494c-53e3-4494-ba23-9743e29f2c99 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:37.623260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 911/1814] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:37.629164 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c03f23-80a6-453b-a028-73018b29d08b admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:37.641173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c03f23-80a6-453b-a028-73018b29d08b admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:37.646619 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03d521-ed22-423e-a307-ce8c280b540a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:37.647328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03d521-ed22-423e-a307-ce8c280b540a admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:37.647610 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03d521-ed22-423e-a307-ce8c280b540a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:37.647889 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03d521-ed22-423e-a307-ce8c280b540a admin admin] PATH_INFO: `/v3/projects/410411390c4d47bfae6d86e4b9e17c4c/users/3c24d92d542647faba893336ff93bfaa/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:37.664636 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03d521-ed22-423e-a307-ce8c280b540a admin admin] RBAC: Authorizing `identity:create_grant(project_id=410411390c4d47bfae6d86e4b9e17c4c, user_id=3c24d92d542647faba893336ff93bfaa, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:37.667699 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03d521-ed22-423e-a307-ce8c280b540a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:37.781166 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 904/1815] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:37 2022] PUT /identity/v3/projects/410411390c4d47bfae6d86e4b9e17c4c/users/3c24d92d542647faba893336ff93bfaa/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:37.785800 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6494c-53e3-4494-ba23-9743e29f2c99 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:37.795879 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6494c-53e3-4494-ba23-9743e29f2c99 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:37.858008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b7a49-dd1e-43b1-826b-9d9eee565cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:37.858616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b7a49-dd1e-43b1-826b-9d9eee565cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:37.858920 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b7a49-dd1e-43b1-826b-9d9eee565cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:37.859203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b7a49-dd1e-43b1-826b-9d9eee565cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:37.859846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b7a49-dd1e-43b1-826b-9d9eee565cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:37.861483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b7a49-dd1e-43b1-826b-9d9eee565cb4 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:37.874713 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 912/1816] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:37.876063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03d521-ed22-423e-a307-ce8c280b540a admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:37.890312 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03d521-ed22-423e-a307-ce8c280b540a admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:37.895664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c8ad2-1524-460b-843c-50362e354c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:37.897480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c8ad2-1524-460b-843c-50362e354c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:37.897480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c8ad2-1524-460b-843c-50362e354c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:37.897480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c8ad2-1524-460b-843c-50362e354c41 admin admin] PATH_INFO: `/v3/projects/410411390c4d47bfae6d86e4b9e17c4c/users/3c24d92d542647faba893336ff93bfaa/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:37.899232 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c8ad2-1524-460b-843c-50362e354c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=410411390c4d47bfae6d86e4b9e17c4c, user_id=3c24d92d542647faba893336ff93bfaa, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:37.901907 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c8ad2-1524-460b-843c-50362e354c41 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:37.917026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 905/1817] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:37 2022] PUT /identity/v3/projects/410411390c4d47bfae6d86e4b9e17c4c/users/3c24d92d542647faba893336ff93bfaa/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:37.926308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43354a26-04db-42ab-8e46-db533cd19b9e None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:37.926617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43354a26-04db-42ab-8e46-db533cd19b9e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:37.926886 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43354a26-04db-42ab-8e46-db533cd19b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:37.958407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-43354a26-04db-42ab-8e46-db533cd19b9e None None] MFA Rules not processed for user `3c24d92d542647faba893336ff93bfaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:37.959895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-43354a26-04db-42ab-8e46-db533cd19b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:37.975834 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:38.004265 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:38.044206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 913/1818] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:37 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:38.055903 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c8ad2-1524-460b-843c-50362e354c41 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:38.057797 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-aa4c8ad2-1524-460b-843c-50362e354c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:38.120883 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c8ad2-1524-460b-843c-50362e354c41 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:38.125580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5d987-c3e0-4444-9455-c9faa147ffeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:38.126212 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5d987-c3e0-4444-9455-c9faa147ffeb service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:38.126579 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5d987-c3e0-4444-9455-c9faa147ffeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:38.126756 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5d987-c3e0-4444-9455-c9faa147ffeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:38.138609 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5d987-c3e0-4444-9455-c9faa147ffeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:38.140782 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5d987-c3e0-4444-9455-c9faa147ffeb service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:38.152386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 906/1819] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:38.266152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43354a26-04db-42ab-8e46-db533cd19b9e None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:38.268358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-43354a26-04db-42ab-8e46-db533cd19b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:38.281090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43354a26-04db-42ab-8e46-db533cd19b9e None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:38.292238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4dba15-fe86-4edd-b374-f16a7b7d2034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:38.292911 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dba15-fe86-4edd-b374-f16a7b7d2034 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:38.293194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dba15-fe86-4edd-b374-f16a7b7d2034 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:38.293467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dba15-fe86-4edd-b374-f16a7b7d2034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:38.354731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4dba15-fe86-4edd-b374-f16a7b7d2034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:38.356261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4dba15-fe86-4edd-b374-f16a7b7d2034 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:38.367308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 914/1820] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:38.878528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:39.427469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:39.866052 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5d987-c3e0-4444-9455-c9faa147ffeb service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:39.868025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-36c5d987-c3e0-4444-9455-c9faa147ffeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:39.949641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5d987-c3e0-4444-9455-c9faa147ffeb service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:39.957589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a95bc-7460-4a02-aede-09594f6d96c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:39.958464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a95bc-7460-4a02-aede-09594f6d96c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:39.958950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a95bc-7460-4a02-aede-09594f6d96c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:39.959383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a95bc-7460-4a02-aede-09594f6d96c8 admin admin] PATH_INFO: `/v3/users/a7fafa17c696443dab0cf762c73a4ad4` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:39.960904 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a95bc-7460-4a02-aede-09594f6d96c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7fafa17c696443dab0cf762c73a4ad4)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:39.967551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a95bc-7460-4a02-aede-09594f6d96c8 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:40.189130 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ee3a95bc-7460-4a02-aede-09594f6d96c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7fafa17c696443dab0cf762c73a4ad4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:40.277326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:40.306846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ee3a95bc-7460-4a02-aede-09594f6d96c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7fafa17c696443dab0cf762c73a4ad4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:40.307528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ee3a95bc-7460-4a02-aede-09594f6d96c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7fafa17c696443dab0cf762c73a4ad4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:40.320924 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ee3a95bc-7460-4a02-aede-09594f6d96c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7fafa17c696443dab0cf762c73a4ad4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:40.321571 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ee3a95bc-7460-4a02-aede-09594f6d96c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7fafa17c696443dab0cf762c73a4ad4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:40.334576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 907/1821] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:39 2022] DELETE /identity/v3/users/a7fafa17c696443dab0cf762c73a4ad4 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:25:40.339231 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c55d22-5e1e-4247-aff2-6ab66a69a8f1 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:40.339526 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c55d22-5e1e-4247-aff2-6ab66a69a8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:40.339811 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c55d22-5e1e-4247-aff2-6ab66a69a8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:40.360658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-08c55d22-5e1e-4247-aff2-6ab66a69a8f1 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:40.361994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-08c55d22-5e1e-4247-aff2-6ab66a69a8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:40.424371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 915/1822] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:40.434508 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a95bc-7460-4a02-aede-09594f6d96c8 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:40.436522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ee3a95bc-7460-4a02-aede-09594f6d96c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:40.494140 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a95bc-7460-4a02-aede-09594f6d96c8 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:40.497647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe15b1-96a4-4d5f-b5b8-c3daefd808a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:40.498760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe15b1-96a4-4d5f-b5b8-c3daefd808a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:40.499064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe15b1-96a4-4d5f-b5b8-c3daefd808a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:40.499342 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe15b1-96a4-4d5f-b5b8-c3daefd808a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:40.508302 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe15b1-96a4-4d5f-b5b8-c3daefd808a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:40.509707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe15b1-96a4-4d5f-b5b8-c3daefd808a6 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:40.519989 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 908/1823] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:40.782100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:41.069090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c55d22-5e1e-4247-aff2-6ab66a69a8f1 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:41.071662 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-08c55d22-5e1e-4247-aff2-6ab66a69a8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:41.114726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c55d22-5e1e-4247-aff2-6ab66a69a8f1 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:41.119969 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43250ea4-fc42-49bd-b97c-6ec29cbe7940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:41.120618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43250ea4-fc42-49bd-b97c-6ec29cbe7940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:41.120917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43250ea4-fc42-49bd-b97c-6ec29cbe7940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:41.121192 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43250ea4-fc42-49bd-b97c-6ec29cbe7940 admin admin] PATH_INFO: `/v3/projects/d5fb9d6a11054d3ba63ae1d8e4a21c62` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:41.122328 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43250ea4-fc42-49bd-b97c-6ec29cbe7940 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5fb9d6a11054d3ba63ae1d8e4a21c62)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:41.124465 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43250ea4-fc42-49bd-b97c-6ec29cbe7940 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:42.181778 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-43250ea4-fc42-49bd-b97c-6ec29cbe7940 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5fb9d6a11054d3ba63ae1d8e4a21c62', 'request_id': 'req-43250ea4-fc42-49bd-b97c-6ec29cbe7940'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:42.188347 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-43250ea4-fc42-49bd-b97c-6ec29cbe7940 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5fb9d6a11054d3ba63ae1d8e4a21c62', 'request_id': 'req-43250ea4-fc42-49bd-b97c-6ec29cbe7940'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:42.512778 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-43250ea4-fc42-49bd-b97c-6ec29cbe7940 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5fb9d6a11054d3ba63ae1d8e4a21c62', 'request_id': 'req-43250ea4-fc42-49bd-b97c-6ec29cbe7940'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:42.525499 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-43250ea4-fc42-49bd-b97c-6ec29cbe7940 admin admin] The token cache is being invalidate because project d5fb9d6a11054d3ba63ae1d8e4a21c62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:42.525499 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-43250ea4-fc42-49bd-b97c-6ec29cbe7940 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:42.525993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-43250ea4-fc42-49bd-b97c-6ec29cbe7940 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:42.527663 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 916/1824] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:41 2022] DELETE /identity/v3/projects/d5fb9d6a11054d3ba63ae1d8e4a21c62 => generated 0 bytes in 1460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:42.533477 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791bdd6-f8e4-4f88-b9a9-b02085ebb748 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:42.533968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791bdd6-f8e4-4f88-b9a9-b02085ebb748 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:42.534192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791bdd6-f8e4-4f88-b9a9-b02085ebb748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:42.555817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-a791bdd6-f8e4-4f88-b9a9-b02085ebb748 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:42.557133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a791bdd6-f8e4-4f88-b9a9-b02085ebb748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:42.618338 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 909/1825] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:25:42.626977 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a362b0-7ac2-4bb0-891a-baa87a4eb4c1 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:42.627270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a362b0-7ac2-4bb0-891a-baa87a4eb4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:42.627553 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a362b0-7ac2-4bb0-891a-baa87a4eb4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:42.649349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-21a362b0-7ac2-4bb0-891a-baa87a4eb4c1 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:42.650483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-21a362b0-7ac2-4bb0-891a-baa87a4eb4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:42.658592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 917/1826] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:42.662891 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791bdd6-f8e4-4f88-b9a9-b02085ebb748 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:42.672860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791bdd6-f8e4-4f88-b9a9-b02085ebb748 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:42.677554 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed26db-4838-4707-889f-eee6271b181b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:42.678157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed26db-4838-4707-889f-eee6271b181b admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:42.678435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed26db-4838-4707-889f-eee6271b181b admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:42.678707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed26db-4838-4707-889f-eee6271b181b admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:42.679417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed26db-4838-4707-889f-eee6271b181b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:25:42.679727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed26db-4838-4707-889f-eee6271b181b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:42.681325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed26db-4838-4707-889f-eee6271b181b admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:42.694173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 910/1827] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:25:42.698664 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a362b0-7ac2-4bb0-891a-baa87a4eb4c1 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:42.708733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a362b0-7ac2-4bb0-891a-baa87a4eb4c1 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:42.713482 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c486b90-1d86-41c9-8c16-cead8f34b255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:42.714064 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c486b90-1d86-41c9-8c16-cead8f34b255 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:42.714343 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c486b90-1d86-41c9-8c16-cead8f34b255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:42.714616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c486b90-1d86-41c9-8c16-cead8f34b255 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:42.715286 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c486b90-1d86-41c9-8c16-cead8f34b255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:42.717933 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c486b90-1d86-41c9-8c16-cead8f34b255 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:42.828410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 918/1828] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:42 2022] POST /identity/v3/projects => generated 357 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:25:42.832800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed26db-4838-4707-889f-eee6271b181b admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:42.844693 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed26db-4838-4707-889f-eee6271b181b admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:42.909444 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d6e0c-ab41-4f7f-9466-8e9322c7baff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:42.910118 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d6e0c-ab41-4f7f-9466-8e9322c7baff admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:42.910400 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d6e0c-ab41-4f7f-9466-8e9322c7baff admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:42.910676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d6e0c-ab41-4f7f-9466-8e9322c7baff admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:42.911347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d6e0c-ab41-4f7f-9466-8e9322c7baff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:42.913258 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d6e0c-ab41-4f7f-9466-8e9322c7baff admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:42.925283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:43.044001 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 911/1829] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:42 2022] POST /identity/v3/users => generated 279 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:43.046279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c486b90-1d86-41c9-8c16-cead8f34b255 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:43.057600 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c486b90-1d86-41c9-8c16-cead8f34b255 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:43.062707 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d1bad-ad32-4f66-b173-a82f71d3e0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:43.063441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d1bad-ad32-4f66-b173-a82f71d3e0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:43.063791 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d1bad-ad32-4f66-b173-a82f71d3e0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:43.064136 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d1bad-ad32-4f66-b173-a82f71d3e0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:43.064859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d1bad-ad32-4f66-b173-a82f71d3e0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:43.066535 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d1bad-ad32-4f66-b173-a82f71d3e0f2 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:43.077682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 919/1830] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:43.095594 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d6e0c-ab41-4f7f-9466-8e9322c7baff admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:43.105680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d6e0c-ab41-4f7f-9466-8e9322c7baff admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:43.110479 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d98558-fb96-47aa-9982-3c73174a5bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:43.111111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d98558-fb96-47aa-9982-3c73174a5bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:43.111392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d98558-fb96-47aa-9982-3c73174a5bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:43.111665 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d98558-fb96-47aa-9982-3c73174a5bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:43.112315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d98558-fb96-47aa-9982-3c73174a5bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:43.113890 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d98558-fb96-47aa-9982-3c73174a5bb4 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:43.126926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 912/1831] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:43.135853 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d1bad-ad32-4f66-b173-a82f71d3e0f2 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:43.148331 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d1bad-ad32-4f66-b173-a82f71d3e0f2 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:43.161317 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d48a1d-368e-450e-9536-2e7e4a1d6fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:43.165106 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d48a1d-368e-450e-9536-2e7e4a1d6fdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:43.173032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d48a1d-368e-450e-9536-2e7e4a1d6fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:43.173353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d48a1d-368e-450e-9536-2e7e4a1d6fdb admin admin] PATH_INFO: `/v3/projects/d4232f4fbd744fad855f04299c91cf78/users/e08b76f698b74b39b0923805178c9c07/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:43.199257 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d48a1d-368e-450e-9536-2e7e4a1d6fdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4232f4fbd744fad855f04299c91cf78, user_id=e08b76f698b74b39b0923805178c9c07, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:43.203122 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d48a1d-368e-450e-9536-2e7e4a1d6fdb admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:43.225770 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:43.289630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 920/1832] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:43 2022] PUT /identity/v3/projects/d4232f4fbd744fad855f04299c91cf78/users/e08b76f698b74b39b0923805178c9c07/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:43.300078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d442cda-4a4b-4cc2-8931-511ec7c5f629 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:43.300349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d442cda-4a4b-4cc2-8931-511ec7c5f629 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:43.300630 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d442cda-4a4b-4cc2-8931-511ec7c5f629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:43.334789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-8d442cda-4a4b-4cc2-8931-511ec7c5f629 None None] MFA Rules not processed for user `e08b76f698b74b39b0923805178c9c07`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:43.336028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8d442cda-4a4b-4cc2-8931-511ec7c5f629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:43.403466 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 913/1833] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:43.407997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d48a1d-368e-450e-9536-2e7e4a1d6fdb admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:43.415601 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:43.476921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d48a1d-368e-450e-9536-2e7e4a1d6fdb admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:43.481633 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c741803-4fd6-49e8-8d7f-520cb7f16ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:43.482253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c741803-4fd6-49e8-8d7f-520cb7f16ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:43.482592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c741803-4fd6-49e8-8d7f-520cb7f16ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:43.482974 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c741803-4fd6-49e8-8d7f-520cb7f16ca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:43.483753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c741803-4fd6-49e8-8d7f-520cb7f16ca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:43.485685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c741803-4fd6-49e8-8d7f-520cb7f16ca1 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:43.602169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 921/1834] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:43 2022] POST /identity/v3/projects => generated 355 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:43.605961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d442cda-4a4b-4cc2-8931-511ec7c5f629 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:43.617507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d442cda-4a4b-4cc2-8931-511ec7c5f629 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:43.680488 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75d1fc-7b9a-4682-a9cc-6f47fd55c30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:43.681113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75d1fc-7b9a-4682-a9cc-6f47fd55c30c admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:43.681398 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75d1fc-7b9a-4682-a9cc-6f47fd55c30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:43.681677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75d1fc-7b9a-4682-a9cc-6f47fd55c30c admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:43.682347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75d1fc-7b9a-4682-a9cc-6f47fd55c30c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:43.684279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75d1fc-7b9a-4682-a9cc-6f47fd55c30c admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:43.799633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:43.805476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 914/1835] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:43 2022] POST /identity/v3/users => generated 284 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:25:43.809599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c741803-4fd6-49e8-8d7f-520cb7f16ca1 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:43.820327 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c741803-4fd6-49e8-8d7f-520cb7f16ca1 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:43.824864 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2796ba9-4b28-46d0-a789-af4fa2e0c209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:43.825491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2796ba9-4b28-46d0-a789-af4fa2e0c209 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:43.825771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2796ba9-4b28-46d0-a789-af4fa2e0c209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:43.826060 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2796ba9-4b28-46d0-a789-af4fa2e0c209 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:43.827087 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2796ba9-4b28-46d0-a789-af4fa2e0c209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:43.828506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2796ba9-4b28-46d0-a789-af4fa2e0c209 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:43.839136 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 922/1836] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:43.843601 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75d1fc-7b9a-4682-a9cc-6f47fd55c30c admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:43.854079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75d1fc-7b9a-4682-a9cc-6f47fd55c30c admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:43.859443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49469567-7024-420a-b03e-955fb520ac01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:43.860101 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49469567-7024-420a-b03e-955fb520ac01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:43.860386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49469567-7024-420a-b03e-955fb520ac01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:43.860662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49469567-7024-420a-b03e-955fb520ac01 admin admin] PATH_INFO: `/v3/projects/a868b0afc4f54f29909fcf67a4880720/users/6c02171ce0bf46b496f1ec9d267ec68b/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:43.875901 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49469567-7024-420a-b03e-955fb520ac01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a868b0afc4f54f29909fcf67a4880720, user_id=6c02171ce0bf46b496f1ec9d267ec68b, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:43.878564 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49469567-7024-420a-b03e-955fb520ac01 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:43.901617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:43.964380 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 915/1837] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:43 2022] PUT /identity/v3/projects/a868b0afc4f54f29909fcf67a4880720/users/6c02171ce0bf46b496f1ec9d267ec68b/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:43.968773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2796ba9-4b28-46d0-a789-af4fa2e0c209 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:43.979303 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2796ba9-4b28-46d0-a789-af4fa2e0c209 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:44.037258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a6abc-3b75-41d0-8e32-fe196fc29cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:44.037872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a6abc-3b75-41d0-8e32-fe196fc29cdb admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:44.040951 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a6abc-3b75-41d0-8e32-fe196fc29cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:44.040951 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a6abc-3b75-41d0-8e32-fe196fc29cdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:44.040951 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a6abc-3b75-41d0-8e32-fe196fc29cdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:44.040951 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a6abc-3b75-41d0-8e32-fe196fc29cdb admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:44.051467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 923/1838] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:44.055831 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49469567-7024-420a-b03e-955fb520ac01 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:44.066648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49469567-7024-420a-b03e-955fb520ac01 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:44.072028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c280c79-52bf-4879-9baf-9d8e6395cdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:44.073388 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c280c79-52bf-4879-9baf-9d8e6395cdbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:44.073532 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c280c79-52bf-4879-9baf-9d8e6395cdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:44.073994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c280c79-52bf-4879-9baf-9d8e6395cdbb admin admin] PATH_INFO: `/v3/projects/a868b0afc4f54f29909fcf67a4880720/users/6c02171ce0bf46b496f1ec9d267ec68b/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:44.077952 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c280c79-52bf-4879-9baf-9d8e6395cdbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a868b0afc4f54f29909fcf67a4880720, user_id=6c02171ce0bf46b496f1ec9d267ec68b, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:44.080692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c280c79-52bf-4879-9baf-9d8e6395cdbb admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:44.094087 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 916/1839] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:44 2022] PUT /identity/v3/projects/a868b0afc4f54f29909fcf67a4880720/users/6c02171ce0bf46b496f1ec9d267ec68b/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:44.103852 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bef8b-c0f1-4fb9-a909-426ae27dfca3 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:44.104150 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bef8b-c0f1-4fb9-a909-426ae27dfca3 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:44.104457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bef8b-c0f1-4fb9-a909-426ae27dfca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:44.134067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-941bef8b-c0f1-4fb9-a909-426ae27dfca3 None None] MFA Rules not processed for user `6c02171ce0bf46b496f1ec9d267ec68b`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:44.135339 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-941bef8b-c0f1-4fb9-a909-426ae27dfca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:44.203591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 924/1840] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:44.215201 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c280c79-52bf-4879-9baf-9d8e6395cdbb admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:44.217557 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4c280c79-52bf-4879-9baf-9d8e6395cdbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:44.278604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c280c79-52bf-4879-9baf-9d8e6395cdbb admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:44.289168 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f7bd8-3d9f-4277-854d-c423db81de5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:44.291206 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f7bd8-3d9f-4277-854d-c423db81de5e service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:44.291724 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f7bd8-3d9f-4277-854d-c423db81de5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:44.292076 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f7bd8-3d9f-4277-854d-c423db81de5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:44.302771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f7bd8-3d9f-4277-854d-c423db81de5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:44.304246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f7bd8-3d9f-4277-854d-c423db81de5e service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:44.314127 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 917/1841] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:44.584249 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bef8b-c0f1-4fb9-a909-426ae27dfca3 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:44.594268 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bef8b-c0f1-4fb9-a909-426ae27dfca3 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:44.598407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815871c0-f64a-42e6-a7c4-e822c4eb3afe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:44.601375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815871c0-f64a-42e6-a7c4-e822c4eb3afe service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:44.601375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815871c0-f64a-42e6-a7c4-e822c4eb3afe service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:44.601375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815871c0-f64a-42e6-a7c4-e822c4eb3afe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:44.614354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f7bd8-3d9f-4277-854d-c423db81de5e service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:44.616740 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-3f9f7bd8-3d9f-4277-854d-c423db81de5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:44.664491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815871c0-f64a-42e6-a7c4-e822c4eb3afe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:44.666318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815871c0-f64a-42e6-a7c4-e822c4eb3afe service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:44.683022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 925/1842] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:44 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:44.700792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f7bd8-3d9f-4277-854d-c423db81de5e service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:44.709466 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911750b-bc72-4df1-b114-44f4835e2079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:44.709656 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911750b-bc72-4df1-b114-44f4835e2079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:44.709946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911750b-bc72-4df1-b114-44f4835e2079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:44.710230 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911750b-bc72-4df1-b114-44f4835e2079 admin admin] PATH_INFO: `/v3/users/3c3da37755134ae2b1ba82803da56f78` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:44.711624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911750b-bc72-4df1-b114-44f4835e2079 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c3da37755134ae2b1ba82803da56f78)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:44.713621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911750b-bc72-4df1-b114-44f4835e2079 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:44.867035 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:44.977221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-1911750b-bc72-4df1-b114-44f4835e2079 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c3da37755134ae2b1ba82803da56f78'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:45.057776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815871c0-f64a-42e6-a7c4-e822c4eb3afe service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:45.059307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-815871c0-f64a-42e6-a7c4-e822c4eb3afe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:45.066725 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-1911750b-bc72-4df1-b114-44f4835e2079 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c3da37755134ae2b1ba82803da56f78'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:45.067399 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-1911750b-bc72-4df1-b114-44f4835e2079 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c3da37755134ae2b1ba82803da56f78'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:45.072605 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815871c0-f64a-42e6-a7c4-e822c4eb3afe service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:45.077260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639125fd-d04d-4c99-9250-2421678c339a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:45.077880 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639125fd-d04d-4c99-9250-2421678c339a service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:45.078168 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639125fd-d04d-4c99-9250-2421678c339a service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:45.078446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639125fd-d04d-4c99-9250-2421678c339a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:45.081335 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-639125fd-d04d-4c99-9250-2421678c339a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:45.081776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-1911750b-bc72-4df1-b114-44f4835e2079 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c3da37755134ae2b1ba82803da56f78'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:45.082470 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-1911750b-bc72-4df1-b114-44f4835e2079 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c3da37755134ae2b1ba82803da56f78'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:45.095368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 918/1843] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:44 2022] DELETE /identity/v3/users/3c3da37755134ae2b1ba82803da56f78 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:45.099638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb160882-4d56-49d7-a11d-06fa07008853 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:45.100001 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb160882-4d56-49d7-a11d-06fa07008853 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:45.100287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb160882-4d56-49d7-a11d-06fa07008853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:45.121011 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-cb160882-4d56-49d7-a11d-06fa07008853 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:45.122631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-cb160882-4d56-49d7-a11d-06fa07008853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:45.175260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639125fd-d04d-4c99-9250-2421678c339a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:45.176792 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639125fd-d04d-4c99-9250-2421678c339a service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:45.177068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:45.189387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 926/1844] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:25:45.208637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 919/1845] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:25:45.218854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639125fd-d04d-4c99-9250-2421678c339a service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:45.220630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-639125fd-d04d-4c99-9250-2421678c339a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:45.279140 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639125fd-d04d-4c99-9250-2421678c339a service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:45.329256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d92c58-dbd1-44d6-9fb3-8eeb085a271a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:45.329864 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d92c58-dbd1-44d6-9fb3-8eeb085a271a service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:45.330147 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d92c58-dbd1-44d6-9fb3-8eeb085a271a service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:45.330423 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d92c58-dbd1-44d6-9fb3-8eeb085a271a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:45.339695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d92c58-dbd1-44d6-9fb3-8eeb085a271a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:45.341129 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d92c58-dbd1-44d6-9fb3-8eeb085a271a service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:45.350709 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 927/1846] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:45.566216 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:45.645591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:45.885839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb160882-4d56-49d7-a11d-06fa07008853 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:45.887438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-cb160882-4d56-49d7-a11d-06fa07008853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:45.901986 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb160882-4d56-49d7-a11d-06fa07008853 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:45.910135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eabf0c-353c-4561-9ded-f837d59517fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:45.911068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eabf0c-353c-4561-9ded-f837d59517fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:45.911489 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eabf0c-353c-4561-9ded-f837d59517fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:45.911931 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eabf0c-353c-4561-9ded-f837d59517fb admin admin] PATH_INFO: `/v3/projects/27e387d970974c54a75e48614fcb993f` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:45.914111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eabf0c-353c-4561-9ded-f837d59517fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=27e387d970974c54a75e48614fcb993f)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:45.916419 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eabf0c-353c-4561-9ded-f837d59517fb admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:46.068651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-86eabf0c-353c-4561-9ded-f837d59517fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27e387d970974c54a75e48614fcb993f', 'request_id': 'req-86eabf0c-353c-4561-9ded-f837d59517fb'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:46.074206 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-86eabf0c-353c-4561-9ded-f837d59517fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27e387d970974c54a75e48614fcb993f', 'request_id': 'req-86eabf0c-353c-4561-9ded-f837d59517fb'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:46.081526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-86eabf0c-353c-4561-9ded-f837d59517fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27e387d970974c54a75e48614fcb993f', 'request_id': 'req-86eabf0c-353c-4561-9ded-f837d59517fb'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:46.111042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:46.221542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-86eabf0c-353c-4561-9ded-f837d59517fb admin admin] The token cache is being invalidate because project 27e387d970974c54a75e48614fcb993f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:46.221833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-86eabf0c-353c-4561-9ded-f837d59517fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:46.222493 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-86eabf0c-353c-4561-9ded-f837d59517fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:46.224110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 920/1847] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:45 2022] DELETE /identity/v3/projects/27e387d970974c54a75e48614fcb993f => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:46.228398 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d92c58-dbd1-44d6-9fb3-8eeb085a271a service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:46.238243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d92c58-dbd1-44d6-9fb3-8eeb085a271a service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:46.296666 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bee4f-55ed-40be-8ad1-ae22a36af329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:46.297229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bee4f-55ed-40be-8ad1-ae22a36af329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:46.297510 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bee4f-55ed-40be-8ad1-ae22a36af329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:46.297790 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bee4f-55ed-40be-8ad1-ae22a36af329 admin admin] PATH_INFO: `/v3/users/3c24d92d542647faba893336ff93bfaa` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:46.298957 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8bee4f-55ed-40be-8ad1-ae22a36af329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c24d92d542647faba893336ff93bfaa)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:46.301429 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8bee4f-55ed-40be-8ad1-ae22a36af329 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:46.484742 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:46.490512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ad8bee4f-55ed-40be-8ad1-ae22a36af329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c24d92d542647faba893336ff93bfaa'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:46.503929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ad8bee4f-55ed-40be-8ad1-ae22a36af329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c24d92d542647faba893336ff93bfaa'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:46.608367 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ad8bee4f-55ed-40be-8ad1-ae22a36af329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c24d92d542647faba893336ff93bfaa'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:46.615983 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ad8bee4f-55ed-40be-8ad1-ae22a36af329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c24d92d542647faba893336ff93bfaa'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:46.616693 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ad8bee4f-55ed-40be-8ad1-ae22a36af329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c24d92d542647faba893336ff93bfaa'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:46.623547 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 928/1848] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:46 2022] DELETE /identity/v3/users/3c24d92d542647faba893336ff93bfaa => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:47.210722 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eabf0c-353c-4561-9ded-f837d59517fb admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:47.212356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-86eabf0c-353c-4561-9ded-f837d59517fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:47.241513 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eabf0c-353c-4561-9ded-f837d59517fb admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:47.313293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7841b-4c58-4f55-bd99-9407a0927053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:47.313949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7841b-4c58-4f55-bd99-9407a0927053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:47.314237 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7841b-4c58-4f55-bd99-9407a0927053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:47.314551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7841b-4c58-4f55-bd99-9407a0927053 admin admin] PATH_INFO: `/v3/projects/410411390c4d47bfae6d86e4b9e17c4c` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:47.315924 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7841b-4c58-4f55-bd99-9407a0927053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=410411390c4d47bfae6d86e4b9e17c4c)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:47.317844 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7841b-4c58-4f55-bd99-9407a0927053 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:47.444482 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c1d7841b-4c58-4f55-bd99-9407a0927053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '410411390c4d47bfae6d86e4b9e17c4c', 'request_id': 'req-c1d7841b-4c58-4f55-bd99-9407a0927053'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.450170 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c1d7841b-4c58-4f55-bd99-9407a0927053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '410411390c4d47bfae6d86e4b9e17c4c', 'request_id': 'req-c1d7841b-4c58-4f55-bd99-9407a0927053'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.456983 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c1d7841b-4c58-4f55-bd99-9407a0927053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '410411390c4d47bfae6d86e4b9e17c4c', 'request_id': 'req-c1d7841b-4c58-4f55-bd99-9407a0927053'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.546777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c1d7841b-4c58-4f55-bd99-9407a0927053 admin admin] The token cache is being invalidate because project 410411390c4d47bfae6d86e4b9e17c4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:47.547100 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c1d7841b-4c58-4f55-bd99-9407a0927053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.547763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c1d7841b-4c58-4f55-bd99-9407a0927053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.549283 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 921/1849] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:47 2022] DELETE /identity/v3/projects/410411390c4d47bfae6d86e4b9e17c4c => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:47.554160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02134a-82b5-4120-b9b2-d419bf6c27b0 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:47.554614 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02134a-82b5-4120-b9b2-d419bf6c27b0 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:47.554702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02134a-82b5-4120-b9b2-d419bf6c27b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:47.574891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-cb02134a-82b5-4120-b9b2-d419bf6c27b0 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:47.576589 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-cb02134a-82b5-4120-b9b2-d419bf6c27b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:47.641917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 929/1850] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:47.651514 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581aa90-05a7-4742-87d1-9606f140b2a1 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:47.651514 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581aa90-05a7-4742-87d1-9606f140b2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:47.651759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581aa90-05a7-4742-87d1-9606f140b2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:47.672420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-8581aa90-05a7-4742-87d1-9606f140b2a1 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:47.673639 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8581aa90-05a7-4742-87d1-9606f140b2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:47.681710 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 922/1851] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:47.686543 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02134a-82b5-4120-b9b2-d419bf6c27b0 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:47.696734 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02134a-82b5-4120-b9b2-d419bf6c27b0 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:47.702041 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9d498-c673-4e3f-93e2-7c367abf791a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:47.702777 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9d498-c673-4e3f-93e2-7c367abf791a admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:47.703085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9d498-c673-4e3f-93e2-7c367abf791a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:47.703363 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9d498-c673-4e3f-93e2-7c367abf791a admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:47.704046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9d498-c673-4e3f-93e2-7c367abf791a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:25:47.704351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9d498-c673-4e3f-93e2-7c367abf791a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:47.705963 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9d498-c673-4e3f-93e2-7c367abf791a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:47.718792 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 930/1852] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:25:47.723240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581aa90-05a7-4742-87d1-9606f140b2a1 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:47.732961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581aa90-05a7-4742-87d1-9606f140b2a1 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:47.737597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9a307-e893-4089-a837-353d69152825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:47.738190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9a307-e893-4089-a837-353d69152825 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:47.738467 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9a307-e893-4089-a837-353d69152825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:47.738736 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9a307-e893-4089-a837-353d69152825 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:47.739525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9a307-e893-4089-a837-353d69152825 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:47.741404 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9a307-e893-4089-a837-353d69152825 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:47.828738 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 923/1853] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:47 2022] POST /identity/v3/projects => generated 391 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:47.833109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9d498-c673-4e3f-93e2-7c367abf791a admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:47.843350 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9d498-c673-4e3f-93e2-7c367abf791a admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:47.901073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88fcb3-4bde-4e32-8a05-e81d700b85da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:47.901750 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88fcb3-4bde-4e32-8a05-e81d700b85da admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:47.901976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88fcb3-4bde-4e32-8a05-e81d700b85da admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:47.902248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88fcb3-4bde-4e32-8a05-e81d700b85da admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:47.902921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88fcb3-4bde-4e32-8a05-e81d700b85da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:47.905228 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88fcb3-4bde-4e32-8a05-e81d700b85da admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.004967 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 931/1854] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:47 2022] POST /identity/v3/users => generated 296 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:48.009081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9a307-e893-4089-a837-353d69152825 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.020411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9a307-e893-4089-a837-353d69152825 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.024916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2b8b3-4c8d-4f17-80a3-1291e08da13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.025587 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2b8b3-4c8d-4f17-80a3-1291e08da13f admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.025871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2b8b3-4c8d-4f17-80a3-1291e08da13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.026163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2b8b3-4c8d-4f17-80a3-1291e08da13f admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.026790 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a2b8b3-4c8d-4f17-80a3-1291e08da13f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.028429 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a2b8b3-4c8d-4f17-80a3-1291e08da13f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.030708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:48.039148 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 924/1855] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:48.043213 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88fcb3-4bde-4e32-8a05-e81d700b85da admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.054045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88fcb3-4bde-4e32-8a05-e81d700b85da admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.061244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607c73e-b97d-432a-9853-bcc9ecaaf273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.061244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607c73e-b97d-432a-9853-bcc9ecaaf273 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.061244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607c73e-b97d-432a-9853-bcc9ecaaf273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.061244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607c73e-b97d-432a-9853-bcc9ecaaf273 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.061595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607c73e-b97d-432a-9853-bcc9ecaaf273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.063267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607c73e-b97d-432a-9853-bcc9ecaaf273 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.074395 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 932/1856] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:48.079436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2b8b3-4c8d-4f17-80a3-1291e08da13f admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.089515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2b8b3-4c8d-4f17-80a3-1291e08da13f admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.096450 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920d553-7d79-4ea2-92c9-5b1b094b61f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.097208 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920d553-7d79-4ea2-92c9-5b1b094b61f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.097665 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920d553-7d79-4ea2-92c9-5b1b094b61f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.098022 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920d553-7d79-4ea2-92c9-5b1b094b61f8 admin admin] PATH_INFO: `/v3/projects/05d1c618b3e4418eb28c373f1d6e98d3/users/fa37ea966ac8455e9e1423e7a099a14b/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.116150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920d553-7d79-4ea2-92c9-5b1b094b61f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05d1c618b3e4418eb28c373f1d6e98d3, user_id=fa37ea966ac8455e9e1423e7a099a14b, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.118929 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920d553-7d79-4ea2-92c9-5b1b094b61f8 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.214654 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 925/1857] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:48 2022] PUT /identity/v3/projects/05d1c618b3e4418eb28c373f1d6e98d3/users/fa37ea966ac8455e9e1423e7a099a14b/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:48.218015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc6514-c34a-4c03-83f0-4c30c61aba78 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.218398 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc6514-c34a-4c03-83f0-4c30c61aba78 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.218739 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc6514-c34a-4c03-83f0-4c30c61aba78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.255241 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-64fc6514-c34a-4c03-83f0-4c30c61aba78 None None] MFA Rules not processed for user `fa37ea966ac8455e9e1423e7a099a14b`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:48.256788 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-64fc6514-c34a-4c03-83f0-4c30c61aba78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:48.268682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:48.353484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 933/1858] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:25:48.358925 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920d553-7d79-4ea2-92c9-5b1b094b61f8 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.446549 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920d553-7d79-4ea2-92c9-5b1b094b61f8 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.452087 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d1f2e-ac1a-406e-811d-916274e0605d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.460669 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d1f2e-ac1a-406e-811d-916274e0605d admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.460669 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d1f2e-ac1a-406e-811d-916274e0605d admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.460669 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d1f2e-ac1a-406e-811d-916274e0605d admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.460669 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d1f2e-ac1a-406e-811d-916274e0605d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.460669 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d1f2e-ac1a-406e-811d-916274e0605d admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.583182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 926/1859] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:48 2022] POST /identity/v3/projects => generated 389 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:48.587165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc6514-c34a-4c03-83f0-4c30c61aba78 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.598220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc6514-c34a-4c03-83f0-4c30c61aba78 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.653970 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d757869-34c5-487f-a2da-fd25e3466ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.654716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d757869-34c5-487f-a2da-fd25e3466ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.655030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d757869-34c5-487f-a2da-fd25e3466ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.655306 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d757869-34c5-487f-a2da-fd25e3466ac5 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.655951 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d757869-34c5-487f-a2da-fd25e3466ac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.657870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d757869-34c5-487f-a2da-fd25e3466ac5 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.731522 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 934/1860] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:48 2022] POST /identity/v3/users => generated 301 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:48.735944 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d1f2e-ac1a-406e-811d-916274e0605d admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.746629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d1f2e-ac1a-406e-811d-916274e0605d admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.751997 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65478a50-63d1-486b-bcc8-fa5a49e942d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.752611 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65478a50-63d1-486b-bcc8-fa5a49e942d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.752890 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65478a50-63d1-486b-bcc8-fa5a49e942d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.753166 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65478a50-63d1-486b-bcc8-fa5a49e942d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.753806 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65478a50-63d1-486b-bcc8-fa5a49e942d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.755457 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65478a50-63d1-486b-bcc8-fa5a49e942d5 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.767545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 927/1861] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:48.771852 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d757869-34c5-487f-a2da-fd25e3466ac5 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.782268 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d757869-34c5-487f-a2da-fd25e3466ac5 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.787050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13a45d-2207-40b7-9f76-5ed4482d9ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.787817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13a45d-2207-40b7-9f76-5ed4482d9ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.788160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13a45d-2207-40b7-9f76-5ed4482d9ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.788500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13a45d-2207-40b7-9f76-5ed4482d9ee6 admin admin] PATH_INFO: `/v3/projects/8dbd217afe5746fdbb8d310cf4b90c66/users/28b75abe3da14b529fb8685f0fad50b5/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.804863 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13a45d-2207-40b7-9f76-5ed4482d9ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dbd217afe5746fdbb8d310cf4b90c66, user_id=28b75abe3da14b529fb8685f0fad50b5, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.816246 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13a45d-2207-40b7-9f76-5ed4482d9ee6 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.901142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 935/1862] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:48 2022] PUT /identity/v3/projects/8dbd217afe5746fdbb8d310cf4b90c66/users/28b75abe3da14b529fb8685f0fad50b5/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:25:48.905221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65478a50-63d1-486b-bcc8-fa5a49e942d5 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.915589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65478a50-63d1-486b-bcc8-fa5a49e942d5 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.977943 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026137c-dd06-44c2-ab63-a136e681e1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.978564 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026137c-dd06-44c2-ab63-a136e681e1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.978848 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026137c-dd06-44c2-ab63-a136e681e1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.979157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026137c-dd06-44c2-ab63-a136e681e1e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.979798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026137c-dd06-44c2-ab63-a136e681e1e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.981541 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026137c-dd06-44c2-ab63-a136e681e1e1 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.992719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 928/1863] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:48.997010 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13a45d-2207-40b7-9f76-5ed4482d9ee6 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:49.008789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13a45d-2207-40b7-9f76-5ed4482d9ee6 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:49.013426 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf862e-ee58-44df-b971-5ef728fc8bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:49.014359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf862e-ee58-44df-b971-5ef728fc8bb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:49.014728 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf862e-ee58-44df-b971-5ef728fc8bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:49.015111 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf862e-ee58-44df-b971-5ef728fc8bb9 admin admin] PATH_INFO: `/v3/projects/8dbd217afe5746fdbb8d310cf4b90c66/users/28b75abe3da14b529fb8685f0fad50b5/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:49.017046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf862e-ee58-44df-b971-5ef728fc8bb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dbd217afe5746fdbb8d310cf4b90c66, user_id=28b75abe3da14b529fb8685f0fad50b5, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:49.019859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf862e-ee58-44df-b971-5ef728fc8bb9 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:49.031677 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 936/1864] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:48 2022] PUT /identity/v3/projects/8dbd217afe5746fdbb8d310cf4b90c66/users/28b75abe3da14b529fb8685f0fad50b5/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:49.041905 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed805e-a1ca-4f8a-a436-0f3e28788557 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:49.042229 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed805e-a1ca-4f8a-a436-0f3e28788557 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:49.042504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed805e-a1ca-4f8a-a436-0f3e28788557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:49.074776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-a7ed805e-a1ca-4f8a-a436-0f3e28788557 None None] MFA Rules not processed for user `28b75abe3da14b529fb8685f0fad50b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:49.076027 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a7ed805e-a1ca-4f8a-a436-0f3e28788557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:49.111484 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:49.155197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 929/1865] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:25:49 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:49.166079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf862e-ee58-44df-b971-5ef728fc8bb9 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:49.170503 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-cedf862e-ee58-44df-b971-5ef728fc8bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:49.228446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf862e-ee58-44df-b971-5ef728fc8bb9 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:49.233918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6b215-462c-4d5e-a64c-6824fdfad3cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:49.234806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6b215-462c-4d5e-a64c-6824fdfad3cc service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:49.235060 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6b215-462c-4d5e-a64c-6824fdfad3cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:49.235198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6b215-462c-4d5e-a64c-6824fdfad3cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:49.245298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6b215-462c-4d5e-a64c-6824fdfad3cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:49.246734 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6b215-462c-4d5e-a64c-6824fdfad3cc service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:49.257377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 937/1866] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:49 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:49.348053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed805e-a1ca-4f8a-a436-0f3e28788557 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:49.359173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed805e-a1ca-4f8a-a436-0f3e28788557 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:49.363060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed346c-3b69-49be-94fd-d153a8511704 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:49.363679 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed346c-3b69-49be-94fd-d153a8511704 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:49.363963 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed346c-3b69-49be-94fd-d153a8511704 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:49.364275 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed346c-3b69-49be-94fd-d153a8511704 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:49.458628 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed346c-3b69-49be-94fd-d153a8511704 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:49.461634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed346c-3b69-49be-94fd-d153a8511704 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:49.473971 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 930/1867] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:49 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:54.708678 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e62957-eab8-4e20-8839-e145b1fe8303 None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:54.709267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e62957-eab8-4e20-8839-e145b1fe8303 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:54.709267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e62957-eab8-4e20-8839-e145b1fe8303 None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:54.710584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 938/1868] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 31 19:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:25:54.718569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4694048-d9e3-42fc-bbb3-177cec73480b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:54.719572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4694048-d9e3-42fc-bbb3-177cec73480b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:54.719914 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4694048-d9e3-42fc-bbb3-177cec73480b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:54.741259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-f4694048-d9e3-42fc-bbb3-177cec73480b None None] MFA Rules not processed for user `d461422ce98442bcbe6b254a8dad5e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:54.742545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f4694048-d9e3-42fc-bbb3-177cec73480b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:54.810898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 931/1869] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 31 19:25:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:54.827571 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e62957-eab8-4e20-8839-e145b1fe8303 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:54.830286 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f4e62957-eab8-4e20-8839-e145b1fe8303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:54.848502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e62957-eab8-4e20-8839-e145b1fe8303 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:54.854368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37202f8f-ef6a-4063-912b-9d9a30d9504e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:54.854999 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37202f8f-ef6a-4063-912b-9d9a30d9504e service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:54.855277 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37202f8f-ef6a-4063-912b-9d9a30d9504e service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:54.855551 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37202f8f-ef6a-4063-912b-9d9a30d9504e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:54.862086 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:54.866116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37202f8f-ef6a-4063-912b-9d9a30d9504e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:54.867725 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37202f8f-ef6a-4063-912b-9d9a30d9504e service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:54.878789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 939/1870] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:04.801196 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=73 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:04.911959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:10.617941 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4694048-d9e3-42fc-bbb3-177cec73480b None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:10.697243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4694048-d9e3-42fc-bbb3-177cec73480b None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:10.707601 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612e09c-629f-4567-8763-f3f496ae3296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:10.709756 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612e09c-629f-4567-8763-f3f496ae3296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:10.710254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612e09c-629f-4567-8763-f3f496ae3296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:10.715125 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612e09c-629f-4567-8763-f3f496ae3296 admin admin] PATH_INFO: `/v3/users/fa37ea966ac8455e9e1423e7a099a14b` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:10.717704 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612e09c-629f-4567-8763-f3f496ae3296 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa37ea966ac8455e9e1423e7a099a14b)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:10.719915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612e09c-629f-4567-8763-f3f496ae3296 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:11.010469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6612e09c-629f-4567-8763-f3f496ae3296 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa37ea966ac8455e9e1423e7a099a14b'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:11.118095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6612e09c-629f-4567-8763-f3f496ae3296 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa37ea966ac8455e9e1423e7a099a14b'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:11.118776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6612e09c-629f-4567-8763-f3f496ae3296 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa37ea966ac8455e9e1423e7a099a14b'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:11.131984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6612e09c-629f-4567-8763-f3f496ae3296 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa37ea966ac8455e9e1423e7a099a14b'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:11.132675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6612e09c-629f-4567-8763-f3f496ae3296 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa37ea966ac8455e9e1423e7a099a14b'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:11.144888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 932/1871] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:10 2022] DELETE /identity/v3/users/fa37ea966ac8455e9e1423e7a099a14b => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:11.156579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edda13-11fd-4bfa-991f-8696509d4bf4 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:11.156579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edda13-11fd-4bfa-991f-8696509d4bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:11.156579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edda13-11fd-4bfa-991f-8696509d4bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:11.169288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-69edda13-11fd-4bfa-991f-8696509d4bf4 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:11.170490 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-69edda13-11fd-4bfa-991f-8696509d4bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:11.235602 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 940/1872] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:11.247652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612e09c-629f-4567-8763-f3f496ae3296 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:11.249103 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-6612e09c-629f-4567-8763-f3f496ae3296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:11.322514 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612e09c-629f-4567-8763-f3f496ae3296 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:11.328306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394c049-9abe-4e23-800d-6c165f3212f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:11.328974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394c049-9abe-4e23-800d-6c165f3212f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:11.329282 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394c049-9abe-4e23-800d-6c165f3212f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:11.329553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394c049-9abe-4e23-800d-6c165f3212f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:11.341565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394c049-9abe-4e23-800d-6c165f3212f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:11.343872 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394c049-9abe-4e23-800d-6c165f3212f9 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:11.355003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 933/1873] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:11.885834 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69edda13-11fd-4bfa-991f-8696509d4bf4 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:11.887931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-69edda13-11fd-4bfa-991f-8696509d4bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:11.907858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69edda13-11fd-4bfa-991f-8696509d4bf4 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:11.918615 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2d679-b676-4f7a-9594-7023afe25a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:11.919310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2d679-b676-4f7a-9594-7023afe25a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:11.919598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2d679-b676-4f7a-9594-7023afe25a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:11.919872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2d679-b676-4f7a-9594-7023afe25a6d admin admin] PATH_INFO: `/v3/projects/05d1c618b3e4418eb28c373f1d6e98d3` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:11.921153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2d679-b676-4f7a-9594-7023afe25a6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=05d1c618b3e4418eb28c373f1d6e98d3)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:11.923279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2d679-b676-4f7a-9594-7023afe25a6d admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:12.057443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-10e2d679-b676-4f7a-9594-7023afe25a6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05d1c618b3e4418eb28c373f1d6e98d3', 'request_id': 'req-10e2d679-b676-4f7a-9594-7023afe25a6d'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:12.063147 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-10e2d679-b676-4f7a-9594-7023afe25a6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05d1c618b3e4418eb28c373f1d6e98d3', 'request_id': 'req-10e2d679-b676-4f7a-9594-7023afe25a6d'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:12.152708 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-10e2d679-b676-4f7a-9594-7023afe25a6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05d1c618b3e4418eb28c373f1d6e98d3', 'request_id': 'req-10e2d679-b676-4f7a-9594-7023afe25a6d'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:12.171344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-10e2d679-b676-4f7a-9594-7023afe25a6d admin admin] The token cache is being invalidate because project 05d1c618b3e4418eb28c373f1d6e98d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:12.171909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-10e2d679-b676-4f7a-9594-7023afe25a6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:12.172349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-10e2d679-b676-4f7a-9594-7023afe25a6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:12.173876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 941/1874] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:11 2022] DELETE /identity/v3/projects/05d1c618b3e4418eb28c373f1d6e98d3 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:12.180147 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394c049-9abe-4e23-800d-6c165f3212f9 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:12.190062 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394c049-9abe-4e23-800d-6c165f3212f9 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:12.210533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:12.250499 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd155cf-96e1-4db6-8149-8fbed66504cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:12.251591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd155cf-96e1-4db6-8149-8fbed66504cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:12.252134 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd155cf-96e1-4db6-8149-8fbed66504cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:12.252426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd155cf-96e1-4db6-8149-8fbed66504cd admin admin] PATH_INFO: `/v3/users/28b75abe3da14b529fb8685f0fad50b5` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:12.254565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd155cf-96e1-4db6-8149-8fbed66504cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=28b75abe3da14b529fb8685f0fad50b5)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:12.256631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd155cf-96e1-4db6-8149-8fbed66504cd admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:12.487431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-2fd155cf-96e1-4db6-8149-8fbed66504cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28b75abe3da14b529fb8685f0fad50b5'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:12.592385 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-2fd155cf-96e1-4db6-8149-8fbed66504cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28b75abe3da14b529fb8685f0fad50b5'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:12.593035 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-2fd155cf-96e1-4db6-8149-8fbed66504cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28b75abe3da14b529fb8685f0fad50b5'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:12.606691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-2fd155cf-96e1-4db6-8149-8fbed66504cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28b75abe3da14b529fb8685f0fad50b5'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:12.607387 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-2fd155cf-96e1-4db6-8149-8fbed66504cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28b75abe3da14b529fb8685f0fad50b5'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:12.621355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 934/1875] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:12 2022] DELETE /identity/v3/users/28b75abe3da14b529fb8685f0fad50b5 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:13.114547 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2d679-b676-4f7a-9594-7023afe25a6d admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:13.117234 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-10e2d679-b676-4f7a-9594-7023afe25a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:13.133669 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2d679-b676-4f7a-9594-7023afe25a6d admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:13.204691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b608d5-5af5-49a7-8609-d2991a75434d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:13.206106 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b608d5-5af5-49a7-8609-d2991a75434d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:13.206106 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b608d5-5af5-49a7-8609-d2991a75434d admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:13.206106 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b608d5-5af5-49a7-8609-d2991a75434d admin admin] PATH_INFO: `/v3/projects/8dbd217afe5746fdbb8d310cf4b90c66` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:13.208342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b608d5-5af5-49a7-8609-d2991a75434d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dbd217afe5746fdbb8d310cf4b90c66)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:13.210798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b608d5-5af5-49a7-8609-d2991a75434d admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:13.302937 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:13.322267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-87b608d5-5af5-49a7-8609-d2991a75434d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dbd217afe5746fdbb8d310cf4b90c66', 'request_id': 'req-87b608d5-5af5-49a7-8609-d2991a75434d'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:13.328327 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-87b608d5-5af5-49a7-8609-d2991a75434d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dbd217afe5746fdbb8d310cf4b90c66', 'request_id': 'req-87b608d5-5af5-49a7-8609-d2991a75434d'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:13.415378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-87b608d5-5af5-49a7-8609-d2991a75434d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dbd217afe5746fdbb8d310cf4b90c66', 'request_id': 'req-87b608d5-5af5-49a7-8609-d2991a75434d'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:13.428197 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-87b608d5-5af5-49a7-8609-d2991a75434d admin admin] The token cache is being invalidate because project 8dbd217afe5746fdbb8d310cf4b90c66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:13.428537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-87b608d5-5af5-49a7-8609-d2991a75434d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:13.429312 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-87b608d5-5af5-49a7-8609-d2991a75434d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:13.431149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 942/1876] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:13 2022] DELETE /identity/v3/projects/8dbd217afe5746fdbb8d310cf4b90c66 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:13.436200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2b106-0df5-4020-8d19-5b25aa52adfc None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:13.436714 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2b106-0df5-4020-8d19-5b25aa52adfc None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:13.436714 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2b106-0df5-4020-8d19-5b25aa52adfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:13.457781 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-66b2b106-0df5-4020-8d19-5b25aa52adfc None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:13.459012 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-66b2b106-0df5-4020-8d19-5b25aa52adfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:13.521714 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 935/1877] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:13.529596 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c59493-61df-45ec-a755-b3c3cbbcac45 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:13.529885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c59493-61df-45ec-a755-b3c3cbbcac45 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:13.530161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c59493-61df-45ec-a755-b3c3cbbcac45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:13.551277 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-51c59493-61df-45ec-a755-b3c3cbbcac45 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:13.552645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-51c59493-61df-45ec-a755-b3c3cbbcac45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:13.560854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 943/1878] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:13.565569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2b106-0df5-4020-8d19-5b25aa52adfc None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:13.576161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2b106-0df5-4020-8d19-5b25aa52adfc None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:13.580782 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e6007-4717-4f79-9a5a-80e7743f105f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:13.581387 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e6007-4717-4f79-9a5a-80e7743f105f admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:13.581672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e6007-4717-4f79-9a5a-80e7743f105f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:13.581949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e6007-4717-4f79-9a5a-80e7743f105f admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:13.582593 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e6007-4717-4f79-9a5a-80e7743f105f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:13.582926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e6007-4717-4f79-9a5a-80e7743f105f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:13.584524 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e6007-4717-4f79-9a5a-80e7743f105f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:13.596606 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 936/1879] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:13.600945 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c59493-61df-45ec-a755-b3c3cbbcac45 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:13.610424 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c59493-61df-45ec-a755-b3c3cbbcac45 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:13.615699 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2939a27-919e-4548-af44-4fbf13b20120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:13.616301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2939a27-919e-4548-af44-4fbf13b20120 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:13.616586 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2939a27-919e-4548-af44-4fbf13b20120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:13.616860 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2939a27-919e-4548-af44-4fbf13b20120 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:13.617508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2939a27-919e-4548-af44-4fbf13b20120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:13.619484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2939a27-919e-4548-af44-4fbf13b20120 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:13.747858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 944/1880] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:13 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:26:13.752084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e6007-4717-4f79-9a5a-80e7743f105f admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:13.763102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e6007-4717-4f79-9a5a-80e7743f105f admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:13.793152 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:13.825351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89a2ee-c477-40da-b915-61c4dd095d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:13.825979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89a2ee-c477-40da-b915-61c4dd095d80 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:13.826257 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89a2ee-c477-40da-b915-61c4dd095d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:13.826530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89a2ee-c477-40da-b915-61c4dd095d80 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:13.827186 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89a2ee-c477-40da-b915-61c4dd095d80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:13.829089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89a2ee-c477-40da-b915-61c4dd095d80 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:13.947817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 937/1881] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:13 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:26:13.952196 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2939a27-919e-4548-af44-4fbf13b20120 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:13.963671 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2939a27-919e-4548-af44-4fbf13b20120 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:13.968633 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d74329-119c-472e-b1fb-280daad1c500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:13.969250 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d74329-119c-472e-b1fb-280daad1c500 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:13.969531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d74329-119c-472e-b1fb-280daad1c500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:13.969806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d74329-119c-472e-b1fb-280daad1c500 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:13.970485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d74329-119c-472e-b1fb-280daad1c500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:13.972667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d74329-119c-472e-b1fb-280daad1c500 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:13.983664 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 945/1882] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:13.988682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89a2ee-c477-40da-b915-61c4dd095d80 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:13.999232 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89a2ee-c477-40da-b915-61c4dd095d80 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:14.005304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c0d0a-f708-4cd1-a212-64229d5565e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:14.005909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c0d0a-f708-4cd1-a212-64229d5565e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:14.006190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c0d0a-f708-4cd1-a212-64229d5565e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:14.006465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c0d0a-f708-4cd1-a212-64229d5565e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:14.007137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c0d0a-f708-4cd1-a212-64229d5565e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:14.008712 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c0d0a-f708-4cd1-a212-64229d5565e9 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:14.020581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 938/1883] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:14.024912 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d74329-119c-472e-b1fb-280daad1c500 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:14.035665 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d74329-119c-472e-b1fb-280daad1c500 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:14.040398 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e8a5d-ea42-4b33-8f52-f7f706521c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:14.041053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e8a5d-ea42-4b33-8f52-f7f706521c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:14.041336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e8a5d-ea42-4b33-8f52-f7f706521c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:14.041613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e8a5d-ea42-4b33-8f52-f7f706521c4a admin admin] PATH_INFO: `/v3/projects/73eee3d3df7d47499bbae29b6edc45c5/users/8ad66bd7909943f782a850845370eccc/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:14.057775 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e8a5d-ea42-4b33-8f52-f7f706521c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=73eee3d3df7d47499bbae29b6edc45c5, user_id=8ad66bd7909943f782a850845370eccc, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:14.060783 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e8a5d-ea42-4b33-8f52-f7f706521c4a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:14.124232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 946/1884] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:14 2022] PUT /identity/v3/projects/73eee3d3df7d47499bbae29b6edc45c5/users/8ad66bd7909943f782a850845370eccc/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:14.135131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de778c-4a0d-49b0-916c-831aef044131 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:14.135416 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de778c-4a0d-49b0-916c-831aef044131 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:14.135714 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de778c-4a0d-49b0-916c-831aef044131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:14.168247 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-a7de778c-4a0d-49b0-916c-831aef044131 None None] MFA Rules not processed for user `8ad66bd7909943f782a850845370eccc`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:14.169710 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a7de778c-4a0d-49b0-916c-831aef044131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:14.261367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 939/1885] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:14.265819 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e8a5d-ea42-4b33-8f52-f7f706521c4a admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:14.336142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e8a5d-ea42-4b33-8f52-f7f706521c4a admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:14.340846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139920f9-4bb3-483e-b51d-3f4e9b6cf588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:14.341459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139920f9-4bb3-483e-b51d-3f4e9b6cf588 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:14.341742 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139920f9-4bb3-483e-b51d-3f4e9b6cf588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:14.342045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139920f9-4bb3-483e-b51d-3f4e9b6cf588 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:14.342681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139920f9-4bb3-483e-b51d-3f4e9b6cf588 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:14.344627 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139920f9-4bb3-483e-b51d-3f4e9b6cf588 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:14.428482 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 947/1886] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:14 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:26:14.432897 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de778c-4a0d-49b0-916c-831aef044131 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:14.444203 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de778c-4a0d-49b0-916c-831aef044131 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:14.507178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fb0ed-0353-442b-a6c5-a43af853d8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:14.507839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fb0ed-0353-442b-a6c5-a43af853d8d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:14.508136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fb0ed-0353-442b-a6c5-a43af853d8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:14.508415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fb0ed-0353-442b-a6c5-a43af853d8d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:14.509069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fb0ed-0353-442b-a6c5-a43af853d8d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:14.511143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fb0ed-0353-442b-a6c5-a43af853d8d6 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:14.625186 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 940/1887] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:14 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:14.626315 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139920f9-4bb3-483e-b51d-3f4e9b6cf588 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:14.639205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139920f9-4bb3-483e-b51d-3f4e9b6cf588 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:14.644712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958e1ae-c971-4afe-8c5b-5d52395945e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:14.645334 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958e1ae-c971-4afe-8c5b-5d52395945e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:14.645609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958e1ae-c971-4afe-8c5b-5d52395945e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:14.645879 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958e1ae-c971-4afe-8c5b-5d52395945e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:14.646527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958e1ae-c971-4afe-8c5b-5d52395945e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:14.648372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958e1ae-c971-4afe-8c5b-5d52395945e6 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:14.661146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 948/1888] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:14.665464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fb0ed-0353-442b-a6c5-a43af853d8d6 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:14.676344 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fb0ed-0353-442b-a6c5-a43af853d8d6 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:14.681045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d68ba-796f-42b6-b0ab-a2793adf46a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:14.681645 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d68ba-796f-42b6-b0ab-a2793adf46a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:14.681925 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d68ba-796f-42b6-b0ab-a2793adf46a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:14.682195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d68ba-796f-42b6-b0ab-a2793adf46a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:14.682842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d68ba-796f-42b6-b0ab-a2793adf46a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:14.684451 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d68ba-796f-42b6-b0ab-a2793adf46a8 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:14.696037 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 941/1889] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:14.699948 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958e1ae-c971-4afe-8c5b-5d52395945e6 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:14.710727 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958e1ae-c971-4afe-8c5b-5d52395945e6 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:14.715273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202da1bf-2d04-4ca5-85de-80477cafb10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:14.715917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202da1bf-2d04-4ca5-85de-80477cafb10e admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:14.716193 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202da1bf-2d04-4ca5-85de-80477cafb10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:14.716461 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202da1bf-2d04-4ca5-85de-80477cafb10e admin admin] PATH_INFO: `/v3/projects/1d79fbbf5b464b168aea6e08ecde32d3/users/eca71356effa41fbb50849f832bf846e/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:14.731855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202da1bf-2d04-4ca5-85de-80477cafb10e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d79fbbf5b464b168aea6e08ecde32d3, user_id=eca71356effa41fbb50849f832bf846e, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:14.734679 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202da1bf-2d04-4ca5-85de-80477cafb10e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:14.803794 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:14.809512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 949/1890] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:14 2022] PUT /identity/v3/projects/1d79fbbf5b464b168aea6e08ecde32d3/users/eca71356effa41fbb50849f832bf846e/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:14.821342 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af4ca9-9f4d-4e55-8823-603ec7b972bd None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:14.821632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af4ca9-9f4d-4e55-8823-603ec7b972bd None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:14.821914 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af4ca9-9f4d-4e55-8823-603ec7b972bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:14.851805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-17af4ca9-9f4d-4e55-8823-603ec7b972bd None None] MFA Rules not processed for user `eca71356effa41fbb50849f832bf846e`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:14.853047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-17af4ca9-9f4d-4e55-8823-603ec7b972bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:14.857096 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:14.917468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 942/1891] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:14.922034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202da1bf-2d04-4ca5-85de-80477cafb10e admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:14.986240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202da1bf-2d04-4ca5-85de-80477cafb10e admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:14.992575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173081f-b85f-4337-acad-ed56765fbee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:14.993199 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173081f-b85f-4337-acad-ed56765fbee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:14.993480 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173081f-b85f-4337-acad-ed56765fbee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:14.993750 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173081f-b85f-4337-acad-ed56765fbee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:14.994392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173081f-b85f-4337-acad-ed56765fbee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:14.996515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173081f-b85f-4337-acad-ed56765fbee4 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:15.101366 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 950/1892] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:14 2022] POST /identity/v3/projects => generated 379 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:15.105528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af4ca9-9f4d-4e55-8823-603ec7b972bd None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:15.116854 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af4ca9-9f4d-4e55-8823-603ec7b972bd None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:15.172427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fccddd-c5b8-46ec-b814-2c47be243963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:15.172955 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fccddd-c5b8-46ec-b814-2c47be243963 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:15.173235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fccddd-c5b8-46ec-b814-2c47be243963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:15.173528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fccddd-c5b8-46ec-b814-2c47be243963 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:15.174151 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fccddd-c5b8-46ec-b814-2c47be243963 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:15.176761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fccddd-c5b8-46ec-b814-2c47be243963 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:15.266611 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 943/1893] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:15 2022] POST /identity/v3/users => generated 296 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:15.271022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173081f-b85f-4337-acad-ed56765fbee4 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:15.282207 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173081f-b85f-4337-acad-ed56765fbee4 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:15.287404 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37674696-bf94-45b7-b0f3-22f4876b512e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:15.288069 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37674696-bf94-45b7-b0f3-22f4876b512e admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:15.288351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37674696-bf94-45b7-b0f3-22f4876b512e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:15.288624 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37674696-bf94-45b7-b0f3-22f4876b512e admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:15.289293 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37674696-bf94-45b7-b0f3-22f4876b512e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:15.290861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37674696-bf94-45b7-b0f3-22f4876b512e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:15.304231 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 951/1894] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:15.308566 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fccddd-c5b8-46ec-b814-2c47be243963 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:15.320593 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fccddd-c5b8-46ec-b814-2c47be243963 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:15.325430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb09172-22d1-4b45-877c-6ed4ccd65daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:15.326091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb09172-22d1-4b45-877c-6ed4ccd65daa admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:15.326374 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb09172-22d1-4b45-877c-6ed4ccd65daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:15.326646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb09172-22d1-4b45-877c-6ed4ccd65daa admin admin] PATH_INFO: `/v3/projects/bdead42565c747438364b221743eeb99/users/e7470ef7af5642c19ce2b8e58b2b38a1/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:15.342439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb09172-22d1-4b45-877c-6ed4ccd65daa admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdead42565c747438364b221743eeb99, user_id=e7470ef7af5642c19ce2b8e58b2b38a1, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:15.345269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb09172-22d1-4b45-877c-6ed4ccd65daa admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:15.433467 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 944/1895] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:15 2022] PUT /identity/v3/projects/bdead42565c747438364b221743eeb99/users/e7470ef7af5642c19ce2b8e58b2b38a1/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:15.437377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37674696-bf94-45b7-b0f3-22f4876b512e admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:15.447439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37674696-bf94-45b7-b0f3-22f4876b512e admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:15.508131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50328fb-3ce1-4a0b-b681-614f0599e542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:15.508944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50328fb-3ce1-4a0b-b681-614f0599e542 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:15.509232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50328fb-3ce1-4a0b-b681-614f0599e542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:15.509506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50328fb-3ce1-4a0b-b681-614f0599e542 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:15.510162 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50328fb-3ce1-4a0b-b681-614f0599e542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:15.511834 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50328fb-3ce1-4a0b-b681-614f0599e542 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:15.525681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 952/1896] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:26:15.531653 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb09172-22d1-4b45-877c-6ed4ccd65daa admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:15.543802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb09172-22d1-4b45-877c-6ed4ccd65daa admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:15.548729 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be1382-da97-415a-9c13-12433c228934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:15.549400 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be1382-da97-415a-9c13-12433c228934 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:15.550909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be1382-da97-415a-9c13-12433c228934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:15.550909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be1382-da97-415a-9c13-12433c228934 admin admin] PATH_INFO: `/v3/projects/bdead42565c747438364b221743eeb99/users/e7470ef7af5642c19ce2b8e58b2b38a1/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:15.552647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be1382-da97-415a-9c13-12433c228934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdead42565c747438364b221743eeb99, user_id=e7470ef7af5642c19ce2b8e58b2b38a1, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:15.555390 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be1382-da97-415a-9c13-12433c228934 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:15.566692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 945/1897] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:15 2022] PUT /identity/v3/projects/bdead42565c747438364b221743eeb99/users/e7470ef7af5642c19ce2b8e58b2b38a1/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:15.576873 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90dec6-aff8-4ec8-8967-aca8a236ed8b None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:15.584960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90dec6-aff8-4ec8-8967-aca8a236ed8b None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:15.584960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90dec6-aff8-4ec8-8967-aca8a236ed8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:15.608572 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-4a90dec6-aff8-4ec8-8967-aca8a236ed8b None None] MFA Rules not processed for user `e7470ef7af5642c19ce2b8e58b2b38a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:15.609835 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4a90dec6-aff8-4ec8-8967-aca8a236ed8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:15.674099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:15.685060 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 953/1898] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:15 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:15.696155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be1382-da97-415a-9c13-12433c228934 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:15.697909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-17be1382-da97-415a-9c13-12433c228934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:15.789811 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be1382-da97-415a-9c13-12433c228934 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:15.793617 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bae46c-c645-4fed-9e31-f621d91e6435 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:15.794279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bae46c-c645-4fed-9e31-f621d91e6435 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:15.794605 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bae46c-c645-4fed-9e31-f621d91e6435 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:15.795017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bae46c-c645-4fed-9e31-f621d91e6435 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:15.804928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bae46c-c645-4fed-9e31-f621d91e6435 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:15.806810 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bae46c-c645-4fed-9e31-f621d91e6435 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:15.813621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:15.820306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 946/1899] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:15 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:15.937949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90dec6-aff8-4ec8-8967-aca8a236ed8b None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:15.939456 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4a90dec6-aff8-4ec8-8967-aca8a236ed8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:15.952063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90dec6-aff8-4ec8-8967-aca8a236ed8b None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:15.955793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6528621-a911-4652-92f2-cbf74a23d083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:15.957121 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6528621-a911-4652-92f2-cbf74a23d083 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:15.957430 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6528621-a911-4652-92f2-cbf74a23d083 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:15.957673 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6528621-a911-4652-92f2-cbf74a23d083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:16.018475 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6528621-a911-4652-92f2-cbf74a23d083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:16.020731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6528621-a911-4652-92f2-cbf74a23d083 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:16.029545 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 954/1900] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:26.025607 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=30 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:32.384409 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bae46c-c645-4fed-9e31-f621d91e6435 service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:32.484152 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bae46c-c645-4fed-9e31-f621d91e6435 service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:32.489730 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c73d4a-c1bc-48bf-902c-8ec55317ae4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:32.490600 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c73d4a-c1bc-48bf-902c-8ec55317ae4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:32.490900 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c73d4a-c1bc-48bf-902c-8ec55317ae4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:32.491234 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c73d4a-c1bc-48bf-902c-8ec55317ae4b admin admin] PATH_INFO: `/v3/users/8ad66bd7909943f782a850845370eccc` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:32.492756 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c73d4a-c1bc-48bf-902c-8ec55317ae4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ad66bd7909943f782a850845370eccc)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:32.495034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c73d4a-c1bc-48bf-902c-8ec55317ae4b admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:32.761766 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-62c73d4a-c1bc-48bf-902c-8ec55317ae4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ad66bd7909943f782a850845370eccc'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:32.867355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-62c73d4a-c1bc-48bf-902c-8ec55317ae4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ad66bd7909943f782a850845370eccc'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:32.868136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-62c73d4a-c1bc-48bf-902c-8ec55317ae4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ad66bd7909943f782a850845370eccc'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:32.889187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-62c73d4a-c1bc-48bf-902c-8ec55317ae4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ad66bd7909943f782a850845370eccc'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:32.889983 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-62c73d4a-c1bc-48bf-902c-8ec55317ae4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ad66bd7909943f782a850845370eccc'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:32.905716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 947/1901] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:32 2022] DELETE /identity/v3/users/8ad66bd7909943f782a850845370eccc => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:32.910684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1da890-9913-482d-8f7f-6858d6c8aefc None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:32.911234 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1da890-9913-482d-8f7f-6858d6c8aefc None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:32.911335 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1da890-9913-482d-8f7f-6858d6c8aefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:32.937189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-9d1da890-9913-482d-8f7f-6858d6c8aefc None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:32.937189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9d1da890-9913-482d-8f7f-6858d6c8aefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:33.001079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 955/1902] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:26:33.015053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c73d4a-c1bc-48bf-902c-8ec55317ae4b admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:33.021275 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-62c73d4a-c1bc-48bf-902c-8ec55317ae4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:33.131688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c73d4a-c1bc-48bf-902c-8ec55317ae4b admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:33.134285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7eca7-9bb6-42ed-b2f4-a27c5ab0fc99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:33.134987 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7eca7-9bb6-42ed-b2f4-a27c5ab0fc99 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:33.135271 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7eca7-9bb6-42ed-b2f4-a27c5ab0fc99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:33.136104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7eca7-9bb6-42ed-b2f4-a27c5ab0fc99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:33.147384 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7eca7-9bb6-42ed-b2f4-a27c5ab0fc99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:33.148840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7eca7-9bb6-42ed-b2f4-a27c5ab0fc99 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:33.162907 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 948/1903] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:33.746072 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1da890-9913-482d-8f7f-6858d6c8aefc None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:33.748080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9d1da890-9913-482d-8f7f-6858d6c8aefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:33.763849 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1da890-9913-482d-8f7f-6858d6c8aefc None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:33.768927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9956c5-68e6-480b-bc46-c44ec32b104c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:33.769579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9956c5-68e6-480b-bc46-c44ec32b104c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:33.769919 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9956c5-68e6-480b-bc46-c44ec32b104c admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:33.770195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9956c5-68e6-480b-bc46-c44ec32b104c admin admin] PATH_INFO: `/v3/projects/73eee3d3df7d47499bbae29b6edc45c5` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:33.781483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9956c5-68e6-480b-bc46-c44ec32b104c admin admin] RBAC: Authorizing `identity:delete_project(project_id=73eee3d3df7d47499bbae29b6edc45c5)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:33.783603 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9956c5-68e6-480b-bc46-c44ec32b104c admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:34.004565 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-5a9956c5-68e6-480b-bc46-c44ec32b104c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73eee3d3df7d47499bbae29b6edc45c5', 'request_id': 'req-5a9956c5-68e6-480b-bc46-c44ec32b104c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:34.004565 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-5a9956c5-68e6-480b-bc46-c44ec32b104c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73eee3d3df7d47499bbae29b6edc45c5', 'request_id': 'req-5a9956c5-68e6-480b-bc46-c44ec32b104c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:34.204089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-5a9956c5-68e6-480b-bc46-c44ec32b104c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73eee3d3df7d47499bbae29b6edc45c5', 'request_id': 'req-5a9956c5-68e6-480b-bc46-c44ec32b104c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:34.227222 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-5a9956c5-68e6-480b-bc46-c44ec32b104c admin admin] The token cache is being invalidate because project 73eee3d3df7d47499bbae29b6edc45c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:34.227222 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-5a9956c5-68e6-480b-bc46-c44ec32b104c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:34.227222 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-5a9956c5-68e6-480b-bc46-c44ec32b104c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:34.229393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 956/1904] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:33 2022] DELETE /identity/v3/projects/73eee3d3df7d47499bbae29b6edc45c5 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:34.233186 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7eca7-9bb6-42ed-b2f4-a27c5ab0fc99 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:34.245157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7eca7-9bb6-42ed-b2f4-a27c5ab0fc99 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:34.266233 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:34.320655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f1747-7341-4e40-b483-3aef230b8cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:34.320989 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f1747-7341-4e40-b483-3aef230b8cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:34.321313 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f1747-7341-4e40-b483-3aef230b8cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:34.321610 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f1747-7341-4e40-b483-3aef230b8cf7 admin admin] PATH_INFO: `/v3/users/eca71356effa41fbb50849f832bf846e` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:34.324727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f1747-7341-4e40-b483-3aef230b8cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca71356effa41fbb50849f832bf846e)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:34.326996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f1747-7341-4e40-b483-3aef230b8cf7 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:34.669259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4a4f1747-7341-4e40-b483-3aef230b8cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca71356effa41fbb50849f832bf846e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:34.900629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4a4f1747-7341-4e40-b483-3aef230b8cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca71356effa41fbb50849f832bf846e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:34.901845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4a4f1747-7341-4e40-b483-3aef230b8cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca71356effa41fbb50849f832bf846e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:34.915386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4a4f1747-7341-4e40-b483-3aef230b8cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca71356effa41fbb50849f832bf846e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:34.916063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4a4f1747-7341-4e40-b483-3aef230b8cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca71356effa41fbb50849f832bf846e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:34.926627 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9956c5-68e6-480b-bc46-c44ec32b104c admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:34.928382 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-5a9956c5-68e6-480b-bc46-c44ec32b104c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:34.929629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 949/1905] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:34 2022] DELETE /identity/v3/users/eca71356effa41fbb50849f832bf846e => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:34.941153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9956c5-68e6-480b-bc46-c44ec32b104c admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:35.010317 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e87338-6322-45be-86b3-69df87d6d9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:35.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e87338-6322-45be-86b3-69df87d6d9a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:35.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e87338-6322-45be-86b3-69df87d6d9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:35.011968 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e87338-6322-45be-86b3-69df87d6d9a9 admin admin] PATH_INFO: `/v3/users/e08b76f698b74b39b0923805178c9c07` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:35.012990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e87338-6322-45be-86b3-69df87d6d9a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e08b76f698b74b39b0923805178c9c07)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:35.015544 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e87338-6322-45be-86b3-69df87d6d9a9 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:35.267239 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:35.443614 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-19e87338-6322-45be-86b3-69df87d6d9a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e08b76f698b74b39b0923805178c9c07'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.461084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-19e87338-6322-45be-86b3-69df87d6d9a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e08b76f698b74b39b0923805178c9c07'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.580618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-19e87338-6322-45be-86b3-69df87d6d9a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e08b76f698b74b39b0923805178c9c07'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.589458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-19e87338-6322-45be-86b3-69df87d6d9a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e08b76f698b74b39b0923805178c9c07'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.590521 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-19e87338-6322-45be-86b3-69df87d6d9a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e08b76f698b74b39b0923805178c9c07'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.597358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 957/1906] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:34 2022] DELETE /identity/v3/users/e08b76f698b74b39b0923805178c9c07 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:35.602537 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245530f0-0b2f-454b-84f4-a59b7614f81b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:35.603021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245530f0-0b2f-454b-84f4-a59b7614f81b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:35.603207 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245530f0-0b2f-454b-84f4-a59b7614f81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:35.628620 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-245530f0-0b2f-454b-84f4-a59b7614f81b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:35.630080 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-245530f0-0b2f-454b-84f4-a59b7614f81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:35.632136 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e87338-6322-45be-86b3-69df87d6d9a9 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:35.633585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-19e87338-6322-45be-86b3-69df87d6d9a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:35.719523 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 950/1907] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:26:35.734110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245530f0-0b2f-454b-84f4-a59b7614f81b None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:35.734567 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e87338-6322-45be-86b3-69df87d6d9a9 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:35.736424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-245530f0-0b2f-454b-84f4-a59b7614f81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:35.741763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f23e0a-e081-4058-b592-88306af52330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:35.742840 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f23e0a-e081-4058-b592-88306af52330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:35.745848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f23e0a-e081-4058-b592-88306af52330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:35.749576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f23e0a-e081-4058-b592-88306af52330 admin admin] PATH_INFO: `/v3/projects/1d79fbbf5b464b168aea6e08ecde32d3` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:35.749576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f23e0a-e081-4058-b592-88306af52330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d79fbbf5b464b168aea6e08ecde32d3)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:35.750517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f23e0a-e081-4058-b592-88306af52330 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:35.807896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:35.821591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245530f0-0b2f-454b-84f4-a59b7614f81b None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:35.825466 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f14de-379d-4dfd-9165-d4c9beb6dca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:35.826114 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f14de-379d-4dfd-9165-d4c9beb6dca3 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:35.826449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f14de-379d-4dfd-9165-d4c9beb6dca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:35.826754 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f14de-379d-4dfd-9165-d4c9beb6dca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:35.838022 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f14de-379d-4dfd-9165-d4c9beb6dca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:35.839569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f14de-379d-4dfd-9165-d4c9beb6dca3 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:35.851587 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 951/1908] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:26:35.923519 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-71f23e0a-e081-4058-b592-88306af52330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d79fbbf5b464b168aea6e08ecde32d3', 'request_id': 'req-71f23e0a-e081-4058-b592-88306af52330'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.932376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-71f23e0a-e081-4058-b592-88306af52330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d79fbbf5b464b168aea6e08ecde32d3', 'request_id': 'req-71f23e0a-e081-4058-b592-88306af52330'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.008039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-71f23e0a-e081-4058-b592-88306af52330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d79fbbf5b464b168aea6e08ecde32d3', 'request_id': 'req-71f23e0a-e081-4058-b592-88306af52330'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.021549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-71f23e0a-e081-4058-b592-88306af52330 admin admin] The token cache is being invalidate because project 1d79fbbf5b464b168aea6e08ecde32d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:36.022193 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-71f23e0a-e081-4058-b592-88306af52330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.022609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-71f23e0a-e081-4058-b592-88306af52330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.024423 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 958/1909] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:35 2022] DELETE /identity/v3/projects/1d79fbbf5b464b168aea6e08ecde32d3 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:36.028607 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f14de-379d-4dfd-9165-d4c9beb6dca3 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:36.041259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f14de-379d-4dfd-9165-d4c9beb6dca3 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:36.101909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a9855-4da4-42f1-83b1-f9eacaa7dace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:36.102471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a9855-4da4-42f1-83b1-f9eacaa7dace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:36.102775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a9855-4da4-42f1-83b1-f9eacaa7dace admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:36.103086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a9855-4da4-42f1-83b1-f9eacaa7dace admin admin] PATH_INFO: `/v3/users/e7470ef7af5642c19ce2b8e58b2b38a1` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:36.104284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a9855-4da4-42f1-83b1-f9eacaa7dace admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7470ef7af5642c19ce2b8e58b2b38a1)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:36.106298 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a9855-4da4-42f1-83b1-f9eacaa7dace admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:36.382355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-786a9855-4da4-42f1-83b1-f9eacaa7dace admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7470ef7af5642c19ce2b8e58b2b38a1'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.431068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f23e0a-e081-4058-b592-88306af52330 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:36.432483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-71f23e0a-e081-4058-b592-88306af52330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:36.447551 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f23e0a-e081-4058-b592-88306af52330 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:36.452305 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268445db-c223-4efe-9bbe-78e2305c3510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:36.453603 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268445db-c223-4efe-9bbe-78e2305c3510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:36.454044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268445db-c223-4efe-9bbe-78e2305c3510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:36.454465 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268445db-c223-4efe-9bbe-78e2305c3510 admin admin] PATH_INFO: `/v3/users/e5e7d2c63a4b48e59ffeb9f153a214c5` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:36.455826 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268445db-c223-4efe-9bbe-78e2305c3510 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5e7d2c63a4b48e59ffeb9f153a214c5)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:36.458931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268445db-c223-4efe-9bbe-78e2305c3510 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:36.523711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-786a9855-4da4-42f1-83b1-f9eacaa7dace admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7470ef7af5642c19ce2b8e58b2b38a1'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.524344 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-786a9855-4da4-42f1-83b1-f9eacaa7dace admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7470ef7af5642c19ce2b8e58b2b38a1'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.537774 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-786a9855-4da4-42f1-83b1-f9eacaa7dace admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7470ef7af5642c19ce2b8e58b2b38a1'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.538443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-786a9855-4da4-42f1-83b1-f9eacaa7dace admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7470ef7af5642c19ce2b8e58b2b38a1'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.551425 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 952/1910] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:36 2022] DELETE /identity/v3/users/e7470ef7af5642c19ce2b8e58b2b38a1 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:36.552239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a9855-4da4-42f1-83b1-f9eacaa7dace admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:36.553651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-786a9855-4da4-42f1-83b1-f9eacaa7dace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:36.622523 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:36.626978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a9855-4da4-42f1-83b1-f9eacaa7dace admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:36.632478 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f7ba2-f9f7-45b9-8fd3-9873b33f3fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:36.633582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f7ba2-f9f7-45b9-8fd3-9873b33f3fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:36.633582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f7ba2-f9f7-45b9-8fd3-9873b33f3fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:36.633742 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f7ba2-f9f7-45b9-8fd3-9873b33f3fd2 admin admin] PATH_INFO: `/v3/projects/d4232f4fbd744fad855f04299c91cf78` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:36.634914 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f7ba2-f9f7-45b9-8fd3-9873b33f3fd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4232f4fbd744fad855f04299c91cf78)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:36.636849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f7ba2-f9f7-45b9-8fd3-9873b33f3fd2 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:36.730230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-268445db-c223-4efe-9bbe-78e2305c3510 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5e7d2c63a4b48e59ffeb9f153a214c5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.743035 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-268445db-c223-4efe-9bbe-78e2305c3510 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5e7d2c63a4b48e59ffeb9f153a214c5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.775127 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-884f7ba2-f9f7-45b9-8fd3-9873b33f3fd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4232f4fbd744fad855f04299c91cf78', 'request_id': 'req-884f7ba2-f9f7-45b9-8fd3-9873b33f3fd2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.781199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-884f7ba2-f9f7-45b9-8fd3-9873b33f3fd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4232f4fbd744fad855f04299c91cf78', 'request_id': 'req-884f7ba2-f9f7-45b9-8fd3-9873b33f3fd2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.787851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-884f7ba2-f9f7-45b9-8fd3-9873b33f3fd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4232f4fbd744fad855f04299c91cf78', 'request_id': 'req-884f7ba2-f9f7-45b9-8fd3-9873b33f3fd2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.819302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-268445db-c223-4efe-9bbe-78e2305c3510 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5e7d2c63a4b48e59ffeb9f153a214c5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.828759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-268445db-c223-4efe-9bbe-78e2305c3510 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5e7d2c63a4b48e59ffeb9f153a214c5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.829467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-268445db-c223-4efe-9bbe-78e2305c3510 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5e7d2c63a4b48e59ffeb9f153a214c5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.836527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 959/1911] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:36 2022] DELETE /identity/v3/users/e5e7d2c63a4b48e59ffeb9f153a214c5 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:36.837589 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268445db-c223-4efe-9bbe-78e2305c3510 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:36.838966 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-268445db-c223-4efe-9bbe-78e2305c3510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:36.878854 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:36.911958 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268445db-c223-4efe-9bbe-78e2305c3510 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:36.914851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-884f7ba2-f9f7-45b9-8fd3-9873b33f3fd2 admin admin] The token cache is being invalidate because project d4232f4fbd744fad855f04299c91cf78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:36.914851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-884f7ba2-f9f7-45b9-8fd3-9873b33f3fd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.914851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-884f7ba2-f9f7-45b9-8fd3-9873b33f3fd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.917621 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47eae67-1190-40da-900d-501e4c0355ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:36.918268 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47eae67-1190-40da-900d-501e4c0355ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:36.918551 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47eae67-1190-40da-900d-501e4c0355ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:36.918832 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47eae67-1190-40da-900d-501e4c0355ef admin admin] PATH_INFO: `/v3/users/b5d01942fa8b48b19cee2003966be417` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:36.921825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 953/1912] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:36 2022] DELETE /identity/v3/projects/d4232f4fbd744fad855f04299c91cf78 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:36.921952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47eae67-1190-40da-900d-501e4c0355ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5d01942fa8b48b19cee2003966be417)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:36.922413 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47eae67-1190-40da-900d-501e4c0355ef admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:36.929106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9460ab-6f00-460a-9776-ead5983ea1ca None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:36.929462 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9460ab-6f00-460a-9776-ead5983ea1ca None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:36.930326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9460ab-6f00-460a-9776-ead5983ea1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:36.953764 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-6e9460ab-6f00-460a-9776-ead5983ea1ca None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:36.955102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-6e9460ab-6f00-460a-9776-ead5983ea1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:36.964934 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 954/1913] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:36.965520 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9460ab-6f00-460a-9776-ead5983ea1ca None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:36.966918 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-6e9460ab-6f00-460a-9776-ead5983ea1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:36.984935 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9460ab-6f00-460a-9776-ead5983ea1ca None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:36.990719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb89ac-0f53-4321-a594-d1ec97f9a32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:36.991408 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb89ac-0f53-4321-a594-d1ec97f9a32c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:36.991688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb89ac-0f53-4321-a594-d1ec97f9a32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:36.991988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb89ac-0f53-4321-a594-d1ec97f9a32c admin admin] PATH_INFO: `/v3/users/6c02171ce0bf46b496f1ec9d267ec68b` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:36.993161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb89ac-0f53-4321-a594-d1ec97f9a32c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c02171ce0bf46b496f1ec9d267ec68b)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:36.995106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb89ac-0f53-4321-a594-d1ec97f9a32c admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:37.200761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-b47eae67-1190-40da-900d-501e4c0355ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5d01942fa8b48b19cee2003966be417'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.213722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-b47eae67-1190-40da-900d-501e4c0355ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5d01942fa8b48b19cee2003966be417'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.275225 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d6cb89ac-0f53-4321-a594-d1ec97f9a32c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c02171ce0bf46b496f1ec9d267ec68b'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.284025 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-b47eae67-1190-40da-900d-501e4c0355ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5d01942fa8b48b19cee2003966be417'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.302768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-b47eae67-1190-40da-900d-501e4c0355ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5d01942fa8b48b19cee2003966be417'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.302768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-b47eae67-1190-40da-900d-501e4c0355ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5d01942fa8b48b19cee2003966be417'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.302768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 960/1914] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:36 2022] DELETE /identity/v3/users/b5d01942fa8b48b19cee2003966be417 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:37.302768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47eae67-1190-40da-900d-501e4c0355ef admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:37.303971 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-b47eae67-1190-40da-900d-501e4c0355ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:37.362265 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47eae67-1190-40da-900d-501e4c0355ef admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:37.366832 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ddabf-3919-471f-a4b7-6080f8457d8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:37.367511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ddabf-3919-471f-a4b7-6080f8457d8e service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.367844 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ddabf-3919-471f-a4b7-6080f8457d8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.368124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ddabf-3919-471f-a4b7-6080f8457d8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.369891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-692ddabf-3919-471f-a4b7-6080f8457d8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:37.414547 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d6cb89ac-0f53-4321-a594-d1ec97f9a32c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c02171ce0bf46b496f1ec9d267ec68b'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.415535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d6cb89ac-0f53-4321-a594-d1ec97f9a32c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c02171ce0bf46b496f1ec9d267ec68b'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.428573 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d6cb89ac-0f53-4321-a594-d1ec97f9a32c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c02171ce0bf46b496f1ec9d267ec68b'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.429257 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d6cb89ac-0f53-4321-a594-d1ec97f9a32c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c02171ce0bf46b496f1ec9d267ec68b'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.435824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ddabf-3919-471f-a4b7-6080f8457d8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:37.437498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ddabf-3919-471f-a4b7-6080f8457d8e service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:37.442417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 955/1915] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:36 2022] DELETE /identity/v3/users/6c02171ce0bf46b496f1ec9d267ec68b => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:37.443365 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb89ac-0f53-4321-a594-d1ec97f9a32c admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:37.444765 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d6cb89ac-0f53-4321-a594-d1ec97f9a32c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:37.448067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 961/1916] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:37.449338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0e297-b3d8-4d83-92a6-1cb63cf44bde None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.449669 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0e297-b3d8-4d83-92a6-1cb63cf44bde None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.450019 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0e297-b3d8-4d83-92a6-1cb63cf44bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.471731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-39a0e297-b3d8-4d83-92a6-1cb63cf44bde None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:37.473674 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-39a0e297-b3d8-4d83-92a6-1cb63cf44bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:37.542010 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb89ac-0f53-4321-a594-d1ec97f9a32c admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:37.549679 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:37.551219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.551686 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.552245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff admin admin] PATH_INFO: `/v3/projects/bdead42565c747438364b221743eeb99` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.554307 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdead42565c747438364b221743eeb99)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:37.556777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:37.558591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 962/1917] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:37.570462 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0e297-b3d8-4d83-92a6-1cb63cf44bde None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:37.572429 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-39a0e297-b3d8-4d83-92a6-1cb63cf44bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:37.630315 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0e297-b3d8-4d83-92a6-1cb63cf44bde None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:37.633938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ba89f-52f0-4216-992b-3bcec352953b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:37.634551 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ba89f-52f0-4216-992b-3bcec352953b service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.634833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ba89f-52f0-4216-992b-3bcec352953b service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.635148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ba89f-52f0-4216-992b-3bcec352953b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.644463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ba89f-52f0-4216-992b-3bcec352953b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:37.645955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ba89f-52f0-4216-992b-3bcec352953b service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:37.655598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 963/1918] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:26:37.712556 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdead42565c747438364b221743eeb99', 'request_id': 'req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.719075 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdead42565c747438364b221743eeb99', 'request_id': 'req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.728701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdead42565c747438364b221743eeb99', 'request_id': 'req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.769401 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:37.815808 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff admin admin] The token cache is being invalidate because project bdead42565c747438364b221743eeb99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:37.817000 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.817790 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.819459 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 956/1919] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:37 2022] DELETE /identity/v3/projects/bdead42565c747438364b221743eeb99 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:37.826334 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494de434-ac60-4ae1-947a-191d2409005f None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.826741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494de434-ac60-4ae1-947a-191d2409005f None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.827009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494de434-ac60-4ae1-947a-191d2409005f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.848885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-494de434-ac60-4ae1-947a-191d2409005f None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:37.850127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-494de434-ac60-4ae1-947a-191d2409005f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:37.891018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:37.895964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:37.952462 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 964/1920] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:37.961067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d956c-05d3-44f6-b2b0-f5846d43218d None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.961368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d956c-05d3-44f6-b2b0-f5846d43218d None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.961696 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d956c-05d3-44f6-b2b0-f5846d43218d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.993346 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:38.000339 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-014d956c-05d3-44f6-b2b0-f5846d43218d None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:38.001823 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-014d956c-05d3-44f6-b2b0-f5846d43218d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:38.011383 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 965/1921] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:38.015482 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d956c-05d3-44f6-b2b0-f5846d43218d None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:38.026911 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d956c-05d3-44f6-b2b0-f5846d43218d None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:38.032764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b95d3-b53c-4661-aa27-1eeee24422ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:38.033466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b95d3-b53c-4661-aa27-1eeee24422ca admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:38.033832 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b95d3-b53c-4661-aa27-1eeee24422ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:38.034141 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b95d3-b53c-4661-aa27-1eeee24422ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:38.035504 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b95d3-b53c-4661-aa27-1eeee24422ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:38.035885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b95d3-b53c-4661-aa27-1eeee24422ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:38.037813 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b95d3-b53c-4661-aa27-1eeee24422ca admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:38.049400 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb7ee4-20f1-4650-b6f4-5266cd0711ff admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:38.056402 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 966/1922] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:38.057490 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b95d3-b53c-4661-aa27-1eeee24422ca admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:38.060026 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-535b95d3-b53c-4661-aa27-1eeee24422ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:38.070576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5313d-d82f-4213-8c1f-190cccca3b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:38.072949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5313d-d82f-4213-8c1f-190cccca3b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:38.073440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5313d-d82f-4213-8c1f-190cccca3b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:38.074098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5313d-d82f-4213-8c1f-190cccca3b16 admin admin] PATH_INFO: `/v3/projects/b0044bfd07484ff29152de409375ac37` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:38.075653 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5313d-d82f-4213-8c1f-190cccca3b16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0044bfd07484ff29152de409375ac37)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:38.076168 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b95d3-b53c-4661-aa27-1eeee24422ca admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:38.081444 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5313d-d82f-4213-8c1f-190cccca3b16 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:38.082320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1ade1-69ae-467d-848f-a939c0cc915f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:38.083084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1ade1-69ae-467d-848f-a939c0cc915f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:38.083363 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1ade1-69ae-467d-848f-a939c0cc915f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:38.083849 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1ade1-69ae-467d-848f-a939c0cc915f admin admin] PATH_INFO: `/v3/projects/a868b0afc4f54f29909fcf67a4880720` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:38.085035 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1ade1-69ae-467d-848f-a939c0cc915f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a868b0afc4f54f29909fcf67a4880720)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:38.087441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1ade1-69ae-467d-848f-a939c0cc915f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:38.275185 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d8d1ade1-69ae-467d-848f-a939c0cc915f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a868b0afc4f54f29909fcf67a4880720', 'request_id': 'req-d8d1ade1-69ae-467d-848f-a939c0cc915f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:38.276050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4ce5313d-d82f-4213-8c1f-190cccca3b16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0044bfd07484ff29152de409375ac37', 'request_id': 'req-4ce5313d-d82f-4213-8c1f-190cccca3b16'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:38.287543 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4ce5313d-d82f-4213-8c1f-190cccca3b16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0044bfd07484ff29152de409375ac37', 'request_id': 'req-4ce5313d-d82f-4213-8c1f-190cccca3b16'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:38.290101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d8d1ade1-69ae-467d-848f-a939c0cc915f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a868b0afc4f54f29909fcf67a4880720', 'request_id': 'req-d8d1ade1-69ae-467d-848f-a939c0cc915f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:38.298124 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4ce5313d-d82f-4213-8c1f-190cccca3b16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0044bfd07484ff29152de409375ac37', 'request_id': 'req-4ce5313d-d82f-4213-8c1f-190cccca3b16'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:38.365257 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d8d1ade1-69ae-467d-848f-a939c0cc915f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a868b0afc4f54f29909fcf67a4880720', 'request_id': 'req-d8d1ade1-69ae-467d-848f-a939c0cc915f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:38.378170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d8d1ade1-69ae-467d-848f-a939c0cc915f admin admin] The token cache is being invalidate because project a868b0afc4f54f29909fcf67a4880720 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:38.378465 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d8d1ade1-69ae-467d-848f-a939c0cc915f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:38.379669 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d8d1ade1-69ae-467d-848f-a939c0cc915f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:38.381329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 967/1923] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:38 2022] DELETE /identity/v3/projects/a868b0afc4f54f29909fcf67a4880720 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:38.382274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1ade1-69ae-467d-848f-a939c0cc915f admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:38.383784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d8d1ade1-69ae-467d-848f-a939c0cc915f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:38.444422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4ce5313d-d82f-4213-8c1f-190cccca3b16 admin admin] The token cache is being invalidate because project b0044bfd07484ff29152de409375ac37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:38.444717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4ce5313d-d82f-4213-8c1f-190cccca3b16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:38.445525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4ce5313d-d82f-4213-8c1f-190cccca3b16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:38.450848 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 957/1924] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:37 2022] DELETE /identity/v3/projects/b0044bfd07484ff29152de409375ac37 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:38.450848 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5313d-d82f-4213-8c1f-190cccca3b16 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:38.451605 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1ade1-69ae-467d-848f-a939c0cc915f admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:38.452324 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4ce5313d-d82f-4213-8c1f-190cccca3b16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:38.515501 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d39a03-81e6-4bce-86bc-239f5ba1d5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:38.516201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d39a03-81e6-4bce-86bc-239f5ba1d5d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:38.516547 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d39a03-81e6-4bce-86bc-239f5ba1d5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:38.516888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d39a03-81e6-4bce-86bc-239f5ba1d5d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:38.518198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d39a03-81e6-4bce-86bc-239f5ba1d5d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:38.520036 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d39a03-81e6-4bce-86bc-239f5ba1d5d0 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:38.549551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5313d-d82f-4213-8c1f-190cccca3b16 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:38.555040 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520c9e0-7ecc-482b-8d7b-35e9c850167e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:38.555684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520c9e0-7ecc-482b-8d7b-35e9c850167e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:38.555990 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520c9e0-7ecc-482b-8d7b-35e9c850167e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:38.556268 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520c9e0-7ecc-482b-8d7b-35e9c850167e admin admin] PATH_INFO: `/v3/projects/b9dca28a00cc42c3bdf64febc9089fb2` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:38.557507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520c9e0-7ecc-482b-8d7b-35e9c850167e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9dca28a00cc42c3bdf64febc9089fb2)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:38.560198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520c9e0-7ecc-482b-8d7b-35e9c850167e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:38.944260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 968/1925] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:38 2022] POST /identity/v3/projects => generated 387 bytes in 562 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:38.946013 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f522cd-1723-49e4-8d5a-654c0a906a8d None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:38.946546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f522cd-1723-49e4-8d5a-654c0a906a8d None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:38.947142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f522cd-1723-49e4-8d5a-654c0a906a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:38.971305 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a1f522cd-1723-49e4-8d5a-654c0a906a8d None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:38.972722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a1f522cd-1723-49e4-8d5a-654c0a906a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:39.036744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 969/1926] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:39.037898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ab45c-21ce-4351-8df4-48a41a035caf None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.038190 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ab45c-21ce-4351-8df4-48a41a035caf None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.038469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ab45c-21ce-4351-8df4-48a41a035caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.066387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-b41ab45c-21ce-4351-8df4-48a41a035caf None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:39.068632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-b41ab45c-21ce-4351-8df4-48a41a035caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:39.076222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8520c9e0-7ecc-482b-8d7b-35e9c850167e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9dca28a00cc42c3bdf64febc9089fb2', 'request_id': 'req-8520c9e0-7ecc-482b-8d7b-35e9c850167e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:39.084098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8520c9e0-7ecc-482b-8d7b-35e9c850167e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9dca28a00cc42c3bdf64febc9089fb2', 'request_id': 'req-8520c9e0-7ecc-482b-8d7b-35e9c850167e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:39.091087 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8520c9e0-7ecc-482b-8d7b-35e9c850167e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9dca28a00cc42c3bdf64febc9089fb2', 'request_id': 'req-8520c9e0-7ecc-482b-8d7b-35e9c850167e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:39.132789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 970/1927] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:39.133815 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ab45c-21ce-4351-8df4-48a41a035caf None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.135137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-b41ab45c-21ce-4351-8df4-48a41a035caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:39.156168 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ab45c-21ce-4351-8df4-48a41a035caf None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.161093 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731c4cc-a47b-4065-8ec8-c0abb0fe79a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.161703 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731c4cc-a47b-4065-8ec8-c0abb0fe79a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.161982 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731c4cc-a47b-4065-8ec8-c0abb0fe79a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.162252 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731c4cc-a47b-4065-8ec8-c0abb0fe79a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.162908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731c4cc-a47b-4065-8ec8-c0abb0fe79a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.164823 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731c4cc-a47b-4065-8ec8-c0abb0fe79a1 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.190493 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8520c9e0-7ecc-482b-8d7b-35e9c850167e admin admin] The token cache is being invalidate because project b9dca28a00cc42c3bdf64febc9089fb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:39.190792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8520c9e0-7ecc-482b-8d7b-35e9c850167e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:39.192280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8520c9e0-7ecc-482b-8d7b-35e9c850167e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:39.193924 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 958/1928] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:38 2022] DELETE /identity/v3/projects/b9dca28a00cc42c3bdf64febc9089fb2 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:39.202100 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d61db1-6ef7-4a8b-8e23-3c927282d45d None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.202100 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d61db1-6ef7-4a8b-8e23-3c927282d45d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.202100 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d61db1-6ef7-4a8b-8e23-3c927282d45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.217150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-99d61db1-6ef7-4a8b-8e23-3c927282d45d None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:39.218376 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-99d61db1-6ef7-4a8b-8e23-3c927282d45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:39.232591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 959/1929] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:39.232591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cf20c-cdc7-4a87-9f74-0dab2cf96d99 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.232591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cf20c-cdc7-4a87-9f74-0dab2cf96d99 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.232591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cf20c-cdc7-4a87-9f74-0dab2cf96d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.251740 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-903cf20c-cdc7-4a87-9f74-0dab2cf96d99 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:39.253799 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-903cf20c-cdc7-4a87-9f74-0dab2cf96d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:39.257399 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 971/1930] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:39 2022] POST /identity/v3/users => generated 294 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:39.258513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731c4cc-a47b-4065-8ec8-c0abb0fe79a1 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.262506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d731c4cc-a47b-4065-8ec8-c0abb0fe79a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:39.264904 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 960/1931] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:39.265584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cf20c-cdc7-4a87-9f74-0dab2cf96d99 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.283102 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731c4cc-a47b-4065-8ec8-c0abb0fe79a1 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.284026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cf20c-cdc7-4a87-9f74-0dab2cf96d99 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.292006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de33ece-b6e3-44b6-b274-5fa681912e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.292670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de33ece-b6e3-44b6-b274-5fa681912e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.292956 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de33ece-b6e3-44b6-b274-5fa681912e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.293227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de33ece-b6e3-44b6-b274-5fa681912e1b admin admin] PATH_INFO: `/v3/users/b1628ca0fb584c6290fa0dfe1cea9136` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.294390 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de33ece-b6e3-44b6-b274-5fa681912e1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1628ca0fb584c6290fa0dfe1cea9136)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.295349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458898e-0dee-434f-b8de-2c042a509839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.296059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458898e-0dee-434f-b8de-2c042a509839 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.296442 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458898e-0dee-434f-b8de-2c042a509839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.296629 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de33ece-b6e3-44b6-b274-5fa681912e1b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.296800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458898e-0dee-434f-b8de-2c042a509839 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.297605 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458898e-0dee-434f-b8de-2c042a509839 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:39.298003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458898e-0dee-434f-b8de-2c042a509839 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.300046 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458898e-0dee-434f-b8de-2c042a509839 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.314062 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 961/1932] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:39.315921 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458898e-0dee-434f-b8de-2c042a509839 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.318154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4458898e-0dee-434f-b8de-2c042a509839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:39.332293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458898e-0dee-434f-b8de-2c042a509839 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.337447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43da76d-09e9-4abf-b492-e1da1aad9159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.338157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43da76d-09e9-4abf-b492-e1da1aad9159 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.339157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43da76d-09e9-4abf-b492-e1da1aad9159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.339157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43da76d-09e9-4abf-b492-e1da1aad9159 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.339837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43da76d-09e9-4abf-b492-e1da1aad9159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.341517 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43da76d-09e9-4abf-b492-e1da1aad9159 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.353567 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:39.356105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 962/1933] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:39.356105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43da76d-09e9-4abf-b492-e1da1aad9159 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.366739 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43da76d-09e9-4abf-b492-e1da1aad9159 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.374287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fcf9b1-8f24-45c7-bb23-3af17d82b5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.375143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcf9b1-8f24-45c7-bb23-3af17d82b5bd admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.375495 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcf9b1-8f24-45c7-bb23-3af17d82b5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.375837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcf9b1-8f24-45c7-bb23-3af17d82b5bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.376565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fcf9b1-8f24-45c7-bb23-3af17d82b5bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:39.376940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fcf9b1-8f24-45c7-bb23-3af17d82b5bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.379256 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fcf9b1-8f24-45c7-bb23-3af17d82b5bd admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.392348 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 963/1934] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:39.393163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fcf9b1-8f24-45c7-bb23-3af17d82b5bd admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.406719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fcf9b1-8f24-45c7-bb23-3af17d82b5bd admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.416679 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c82429-4604-4eba-878c-02ba0bfd4791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.416679 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c82429-4604-4eba-878c-02ba0bfd4791 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.416679 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c82429-4604-4eba-878c-02ba0bfd4791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.416679 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c82429-4604-4eba-878c-02ba0bfd4791 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.417082 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c82429-4604-4eba-878c-02ba0bfd4791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.417082 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c82429-4604-4eba-878c-02ba0bfd4791 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.505037 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 964/1935] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:39 2022] POST /identity/v3/projects => generated 363 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:39.505037 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c82429-4604-4eba-878c-02ba0bfd4791 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.553014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7de33ece-b6e3-44b6-b274-5fa681912e1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1628ca0fb584c6290fa0dfe1cea9136'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:39.573674 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7de33ece-b6e3-44b6-b274-5fa681912e1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1628ca0fb584c6290fa0dfe1cea9136'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:39.575015 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c82429-4604-4eba-878c-02ba0bfd4791 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.581136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f73564-22bc-43e2-ab80-166f963cf193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.581854 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f73564-22bc-43e2-ab80-166f963cf193 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.582658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f73564-22bc-43e2-ab80-166f963cf193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.583108 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f73564-22bc-43e2-ab80-166f963cf193 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.583766 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f73564-22bc-43e2-ab80-166f963cf193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.585493 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f73564-22bc-43e2-ab80-166f963cf193 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.596004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 965/1936] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:39.600629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f73564-22bc-43e2-ab80-166f963cf193 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.608223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f73564-22bc-43e2-ab80-166f963cf193 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.615454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eec6d1-a72b-436d-a6d6-6720b9030da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.616093 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eec6d1-a72b-436d-a6d6-6720b9030da7 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.616378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eec6d1-a72b-436d-a6d6-6720b9030da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.616682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eec6d1-a72b-436d-a6d6-6720b9030da7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.617579 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eec6d1-a72b-436d-a6d6-6720b9030da7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.619654 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eec6d1-a72b-436d-a6d6-6720b9030da7 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.662056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7de33ece-b6e3-44b6-b274-5fa681912e1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1628ca0fb584c6290fa0dfe1cea9136'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:39.678895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7de33ece-b6e3-44b6-b274-5fa681912e1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1628ca0fb584c6290fa0dfe1cea9136'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:39.678895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7de33ece-b6e3-44b6-b274-5fa681912e1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1628ca0fb584c6290fa0dfe1cea9136'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:39.679237 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:39.691010 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 972/1937] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:39 2022] DELETE /identity/v3/users/b1628ca0fb584c6290fa0dfe1cea9136 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:39.691882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de33ece-b6e3-44b6-b274-5fa681912e1b admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.693304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7de33ece-b6e3-44b6-b274-5fa681912e1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:39.738266 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 966/1938] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:39 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:39.743199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eec6d1-a72b-436d-a6d6-6720b9030da7 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.745082 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-09eec6d1-a72b-436d-a6d6-6720b9030da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:39.773909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de33ece-b6e3-44b6-b274-5fa681912e1b admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.805435 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:39.837388 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eec6d1-a72b-436d-a6d6-6720b9030da7 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.844434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7c6b0-d353-4b2b-af99-57d52d7eaa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.845395 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7c6b0-d353-4b2b-af99-57d52d7eaa42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.845760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7c6b0-d353-4b2b-af99-57d52d7eaa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.846044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7c6b0-d353-4b2b-af99-57d52d7eaa42 admin admin] PATH_INFO: `/v3/projects/506f0d9adb804e79885908fd3a0c8665/users/ccae720662b64207b9982808334ceb4c/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.876345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da7c6b0-d353-4b2b-af99-57d52d7eaa42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=506f0d9adb804e79885908fd3a0c8665, user_id=ccae720662b64207b9982808334ceb4c, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.878243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da7c6b0-d353-4b2b-af99-57d52d7eaa42 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.932733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6cc68-bad5-4779-88e4-06e9213be91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.933310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6cc68-bad5-4779-88e4-06e9213be91b admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.934182 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6cc68-bad5-4779-88e4-06e9213be91b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.934459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6cc68-bad5-4779-88e4-06e9213be91b admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.935134 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6cc68-bad5-4779-88e4-06e9213be91b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.939970 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6cc68-bad5-4779-88e4-06e9213be91b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.995485 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 967/1939] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:39 2022] PUT /identity/v3/projects/506f0d9adb804e79885908fd3a0c8665/users/ccae720662b64207b9982808334ceb4c/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:39.996664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7c6b0-d353-4b2b-af99-57d52d7eaa42 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:40.008438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-6da7c6b0-d353-4b2b-af99-57d52d7eaa42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:40.086630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 973/1940] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:39 2022] POST /identity/v3/users => generated 282 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:40.086630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3a051-a7e5-4594-b217-fd91225c88bb None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:40.086630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3a051-a7e5-4594-b217-fd91225c88bb None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:40.086630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3a051-a7e5-4594-b217-fd91225c88bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:40.122513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d4e3a051-a7e5-4594-b217-fd91225c88bb None None] MFA Rules not processed for user `ccae720662b64207b9982808334ceb4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:40.122513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d4e3a051-a7e5-4594-b217-fd91225c88bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:40.136469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7c6b0-d353-4b2b-af99-57d52d7eaa42 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:40.143791 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4f0d3-14ee-4b5b-85a0-f47abbc3bcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:40.145579 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4f0d3-14ee-4b5b-85a0-f47abbc3bcef admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:40.145916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4f0d3-14ee-4b5b-85a0-f47abbc3bcef admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:40.146195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4f0d3-14ee-4b5b-85a0-f47abbc3bcef admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:40.147324 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4f0d3-14ee-4b5b-85a0-f47abbc3bcef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:40.149458 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4f0d3-14ee-4b5b-85a0-f47abbc3bcef admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:40.191014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 974/1941] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:40.193013 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3a051-a7e5-4594-b217-fd91225c88bb None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:40.208498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3a051-a7e5-4594-b217-fd91225c88bb None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:40.209394 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5f0ca-2348-43fe-9908-2ebbf10d9df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:40.210003 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5f0ca-2348-43fe-9908-2ebbf10d9df2 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:40.210282 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5f0ca-2348-43fe-9908-2ebbf10d9df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:40.210553 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5f0ca-2348-43fe-9908-2ebbf10d9df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:40.213680 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5f0ca-2348-43fe-9908-2ebbf10d9df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:40.215516 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5f0ca-2348-43fe-9908-2ebbf10d9df2 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:40.228445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 975/1942] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:40.229320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5f0ca-2348-43fe-9908-2ebbf10d9df2 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:40.243083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5f0ca-2348-43fe-9908-2ebbf10d9df2 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:40.244707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 968/1943] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:39 2022] POST /identity/v3/users => generated 286 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:40.246388 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4f0d3-14ee-4b5b-85a0-f47abbc3bcef admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:40.251454 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a3334-4a22-4cf9-a86a-ab01448f947e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:40.252824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a3334-4a22-4cf9-a86a-ab01448f947e admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:40.253239 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a3334-4a22-4cf9-a86a-ab01448f947e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:40.254718 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a3334-4a22-4cf9-a86a-ab01448f947e admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:40.255568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a3334-4a22-4cf9-a86a-ab01448f947e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:40.258080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a3334-4a22-4cf9-a86a-ab01448f947e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:40.261841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4f0d3-14ee-4b5b-85a0-f47abbc3bcef admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:40.268149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed845e3b-2f0f-4886-bc97-16769e11288d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:40.268863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed845e3b-2f0f-4886-bc97-16769e11288d admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:40.269155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed845e3b-2f0f-4886-bc97-16769e11288d admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:40.275516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed845e3b-2f0f-4886-bc97-16769e11288d admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:40.275516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed845e3b-2f0f-4886-bc97-16769e11288d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:40.275516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed845e3b-2f0f-4886-bc97-16769e11288d admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:40.287902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 969/1944] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:40.287902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed845e3b-2f0f-4886-bc97-16769e11288d admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:40.299590 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed845e3b-2f0f-4886-bc97-16769e11288d admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:40.305688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007149f-c4b8-4004-91d5-635f76604374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:40.306316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007149f-c4b8-4004-91d5-635f76604374 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:40.306602 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007149f-c4b8-4004-91d5-635f76604374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:40.306897 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007149f-c4b8-4004-91d5-635f76604374 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:40.307562 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007149f-c4b8-4004-91d5-635f76604374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:40.309228 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007149f-c4b8-4004-91d5-635f76604374 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:40.323579 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 970/1945] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:40.324416 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007149f-c4b8-4004-91d5-635f76604374 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:40.335432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007149f-c4b8-4004-91d5-635f76604374 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:40.340410 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd8501c-dbf2-49e1-b01d-d86f124fd0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:40.341065 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8501c-dbf2-49e1-b01d-d86f124fd0c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:40.341345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8501c-dbf2-49e1-b01d-d86f124fd0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:40.341619 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8501c-dbf2-49e1-b01d-d86f124fd0c9 admin admin] PATH_INFO: `/v3/projects/1243a90f5170471fa3f752e6176bc078/users/988a3046b0f842b3b4dce1891217b0f6/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:40.357692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd8501c-dbf2-49e1-b01d-d86f124fd0c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1243a90f5170471fa3f752e6176bc078, user_id=988a3046b0f842b3b4dce1891217b0f6, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:40.360471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd8501c-dbf2-49e1-b01d-d86f124fd0c9 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:40.377031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 976/1946] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:40 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:40.379621 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a3334-4a22-4cf9-a86a-ab01448f947e admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:40.449241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 971/1947] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:40 2022] PUT /identity/v3/projects/1243a90f5170471fa3f752e6176bc078/users/988a3046b0f842b3b4dce1891217b0f6/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:40.450772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd8501c-dbf2-49e1-b01d-d86f124fd0c9 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:40.453400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a3334-4a22-4cf9-a86a-ab01448f947e admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:40.522541 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61c6ee-3e20-438a-bda9-78d368687549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:40.523218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61c6ee-3e20-438a-bda9-78d368687549 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:40.523498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61c6ee-3e20-438a-bda9-78d368687549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:40.523772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61c6ee-3e20-438a-bda9-78d368687549 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:40.525018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61c6ee-3e20-438a-bda9-78d368687549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:40.526915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61c6ee-3e20-438a-bda9-78d368687549 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:40.562748 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd8501c-dbf2-49e1-b01d-d86f124fd0c9 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:40.571919 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 977/1948] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:40.572086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ede3ce-f32f-4292-8750-17c74a8f24f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:40.572086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ede3ce-f32f-4292-8750-17c74a8f24f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:40.572274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb635b13-78ac-4192-8a25-e586805f72c5 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:40.572660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ede3ce-f32f-4292-8750-17c74a8f24f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:40.573037 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ede3ce-f32f-4292-8750-17c74a8f24f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:40.573362 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb635b13-78ac-4192-8a25-e586805f72c5 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:40.573638 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb635b13-78ac-4192-8a25-e586805f72c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:40.573957 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ede3ce-f32f-4292-8750-17c74a8f24f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:40.576874 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ede3ce-f32f-4292-8750-17c74a8f24f7 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:40.638054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-eb635b13-78ac-4192-8a25-e586805f72c5 None None] MFA Rules not processed for user `988a3046b0f842b3b4dce1891217b0f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:40.639662 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-eb635b13-78ac-4192-8a25-e586805f72c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:40.710822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 972/1949] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:40 2022] POST /identity/v3/users => generated 299 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:40.710822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ede3ce-f32f-4292-8750-17c74a8f24f7 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:40.712137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-31ede3ce-f32f-4292-8750-17c74a8f24f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:40.757612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ede3ce-f32f-4292-8750-17c74a8f24f7 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:40.783903 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3681b86-156d-43ed-8f43-77507eda8613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:40.786483 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3681b86-156d-43ed-8f43-77507eda8613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:40.786768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3681b86-156d-43ed-8f43-77507eda8613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:40.788378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3681b86-156d-43ed-8f43-77507eda8613 admin admin] PATH_INFO: `/v3/projects/57beea4099084ca89630de811752d15b` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:40.791496 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3681b86-156d-43ed-8f43-77507eda8613 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57beea4099084ca89630de811752d15b)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:40.804716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3681b86-156d-43ed-8f43-77507eda8613 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:40.804936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 978/1950] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:40.804936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb635b13-78ac-4192-8a25-e586805f72c5 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:40.816240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb635b13-78ac-4192-8a25-e586805f72c5 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:40.823439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ba3d9-1fe4-4570-b91e-66ec9057e093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:40.824310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ba3d9-1fe4-4570-b91e-66ec9057e093 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:40.824700 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ba3d9-1fe4-4570-b91e-66ec9057e093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:40.825203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ba3d9-1fe4-4570-b91e-66ec9057e093 admin admin] PATH_INFO: `/v3/projects/75e71c290eac4fe393eec1d1d8ee2a59/users/2fb4a631005d4a00aa942d6652ba35c0/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:40.848781 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ba3d9-1fe4-4570-b91e-66ec9057e093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75e71c290eac4fe393eec1d1d8ee2a59, user_id=2fb4a631005d4a00aa942d6652ba35c0, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:40.852083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ba3d9-1fe4-4570-b91e-66ec9057e093 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:40.930894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 979/1951] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:40 2022] PUT /identity/v3/projects/75e71c290eac4fe393eec1d1d8ee2a59/users/2fb4a631005d4a00aa942d6652ba35c0/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:40.931777 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ba3d9-1fe4-4570-b91e-66ec9057e093 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:41.006684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-f3681b86-156d-43ed-8f43-77507eda8613 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57beea4099084ca89630de811752d15b', 'request_id': 'req-f3681b86-156d-43ed-8f43-77507eda8613'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:41.026243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-f3681b86-156d-43ed-8f43-77507eda8613 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57beea4099084ca89630de811752d15b', 'request_id': 'req-f3681b86-156d-43ed-8f43-77507eda8613'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:41.038155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-f3681b86-156d-43ed-8f43-77507eda8613 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57beea4099084ca89630de811752d15b', 'request_id': 'req-f3681b86-156d-43ed-8f43-77507eda8613'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:41.040564 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ba3d9-1fe4-4570-b91e-66ec9057e093 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:41.116148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70237ea4-6494-43dc-a8d4-de34b45414ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:41.116772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70237ea4-6494-43dc-a8d4-de34b45414ae admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:41.117078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70237ea4-6494-43dc-a8d4-de34b45414ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:41.117359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70237ea4-6494-43dc-a8d4-de34b45414ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:41.118028 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70237ea4-6494-43dc-a8d4-de34b45414ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:41.119689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70237ea4-6494-43dc-a8d4-de34b45414ae admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:41.132137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-f3681b86-156d-43ed-8f43-77507eda8613 admin admin] The token cache is being invalidate because project 57beea4099084ca89630de811752d15b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:41.132137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-f3681b86-156d-43ed-8f43-77507eda8613 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:41.132137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-f3681b86-156d-43ed-8f43-77507eda8613 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:41.132137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 973/1952] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:40 2022] DELETE /identity/v3/projects/57beea4099084ca89630de811752d15b => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:41.149915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 980/1953] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:41.149915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd228ed8-0c9e-4f6a-a423-d22e0543fc0e None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:41.149915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd228ed8-0c9e-4f6a-a423-d22e0543fc0e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:41.149915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd228ed8-0c9e-4f6a-a423-d22e0543fc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:41.154062 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3681b86-156d-43ed-8f43-77507eda8613 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:41.164077 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f3681b86-156d-43ed-8f43-77507eda8613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:41.184047 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-cd228ed8-0c9e-4f6a-a423-d22e0543fc0e None None] MFA Rules not processed for user `2fb4a631005d4a00aa942d6652ba35c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:41.188208 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-cd228ed8-0c9e-4f6a-a423-d22e0543fc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:41.190300 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3681b86-156d-43ed-8f43-77507eda8613 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:41.198382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2d3c0-9de8-4728-abab-a372a3ff2137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:41.198916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2d3c0-9de8-4728-abab-a372a3ff2137 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:41.199199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2d3c0-9de8-4728-abab-a372a3ff2137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:41.199470 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2d3c0-9de8-4728-abab-a372a3ff2137 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:41.200104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2d3c0-9de8-4728-abab-a372a3ff2137 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:41.202033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2d3c0-9de8-4728-abab-a372a3ff2137 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:41.250019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:41.301835 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 981/1954] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:41.304576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225bb61-aea7-4d77-9c0f-b0c62557f2e2 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:41.304954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225bb61-aea7-4d77-9c0f-b0c62557f2e2 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:41.305332 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225bb61-aea7-4d77-9c0f-b0c62557f2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:41.328029 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-f225bb61-aea7-4d77-9c0f-b0c62557f2e2 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:41.329628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f225bb61-aea7-4d77-9c0f-b0c62557f2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:41.344866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 982/1955] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:41.345933 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225bb61-aea7-4d77-9c0f-b0c62557f2e2 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:41.347786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f225bb61-aea7-4d77-9c0f-b0c62557f2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:41.350685 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 974/1956] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:41 2022] POST /identity/v3/projects => generated 363 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:41.351986 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2d3c0-9de8-4728-abab-a372a3ff2137 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:41.357148 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b1d2d3c0-9de8-4728-abab-a372a3ff2137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:41.363452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225bb61-aea7-4d77-9c0f-b0c62557f2e2 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:41.433984 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:41.440324 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971477a6-27c0-4f3c-94ce-7cebbf35de42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:41.440324 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971477a6-27c0-4f3c-94ce-7cebbf35de42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:41.440324 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971477a6-27c0-4f3c-94ce-7cebbf35de42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:41.440324 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971477a6-27c0-4f3c-94ce-7cebbf35de42 admin admin] PATH_INFO: `/v3/projects/27e3379e9d764f23a3cd33a8af6c7dac/users/2bd61feb0bbf4022b2e292be10ba3dea/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:41.465760 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971477a6-27c0-4f3c-94ce-7cebbf35de42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27e3379e9d764f23a3cd33a8af6c7dac, user_id=2bd61feb0bbf4022b2e292be10ba3dea, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:41.468627 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971477a6-27c0-4f3c-94ce-7cebbf35de42 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:41.531386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2d3c0-9de8-4728-abab-a372a3ff2137 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:41.539105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91930b37-d39d-462e-bcda-8ce280277387 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f281d778f1404984b4d778a1c26aef1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:41.540056 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91930b37-d39d-462e-bcda-8ce280277387 service glance] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:41.540364 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91930b37-d39d-462e-bcda-8ce280277387 service glance] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:41.540660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91930b37-d39d-462e-bcda-8ce280277387 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:41.558513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 983/1957] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:41 2022] PUT /identity/v3/projects/27e3379e9d764f23a3cd33a8af6c7dac/users/2bd61feb0bbf4022b2e292be10ba3dea/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:41.560119 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3944e7-7ef4-4865-b2b1-636f5adfc6bb None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:41.560808 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3944e7-7ef4-4865-b2b1-636f5adfc6bb None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:41.560808 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3944e7-7ef4-4865-b2b1-636f5adfc6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:41.588324 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-2e3944e7-7ef4-4865-b2b1-636f5adfc6bb None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:41.589759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-2e3944e7-7ef4-4865-b2b1-636f5adfc6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:41.661788 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91930b37-d39d-462e-bcda-8ce280277387 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:41.663333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91930b37-d39d-462e-bcda-8ce280277387 service glance] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:41.675380 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 975/1958] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:41.692179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91930b37-d39d-462e-bcda-8ce280277387 service glance] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:41.721993 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91930b37-d39d-462e-bcda-8ce280277387 service glance] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:41.747893 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 984/1959] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:41.747893 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3944e7-7ef4-4865-b2b1-636f5adfc6bb None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:41.773994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3944e7-7ef4-4865-b2b1-636f5adfc6bb None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:41.788789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301701a2-60fa-4b97-87b1-b96bbebdbf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:41.789597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301701a2-60fa-4b97-87b1-b96bbebdbf5f admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:41.789884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301701a2-60fa-4b97-87b1-b96bbebdbf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:41.790222 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301701a2-60fa-4b97-87b1-b96bbebdbf5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:41.790935 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301701a2-60fa-4b97-87b1-b96bbebdbf5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:41.792523 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301701a2-60fa-4b97-87b1-b96bbebdbf5f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:41.835000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 985/1960] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:41.840177 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301701a2-60fa-4b97-87b1-b96bbebdbf5f admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:41.878327 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301701a2-60fa-4b97-87b1-b96bbebdbf5f admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:41.887701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4721f3-b0d6-4555-95ec-459bb4bb758b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:41.888505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4721f3-b0d6-4555-95ec-459bb4bb758b admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:41.889792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4721f3-b0d6-4555-95ec-459bb4bb758b admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:41.890561 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4721f3-b0d6-4555-95ec-459bb4bb758b admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:41.892386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4721f3-b0d6-4555-95ec-459bb4bb758b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:41.897513 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4721f3-b0d6-4555-95ec-459bb4bb758b admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:41.898164 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246de872-b0a5-44ab-b8e7-4a62c22ccaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:41.903298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246de872-b0a5-44ab-b8e7-4a62c22ccaa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:41.903298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246de872-b0a5-44ab-b8e7-4a62c22ccaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:41.903298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246de872-b0a5-44ab-b8e7-4a62c22ccaa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:41.903298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246de872-b0a5-44ab-b8e7-4a62c22ccaa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:41.903298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246de872-b0a5-44ab-b8e7-4a62c22ccaa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:41.908583 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246de872-b0a5-44ab-b8e7-4a62c22ccaa7 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:41.926712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 986/1961] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:41.932069 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246de872-b0a5-44ab-b8e7-4a62c22ccaa7 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:41.939162 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246de872-b0a5-44ab-b8e7-4a62c22ccaa7 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:41.962203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e1cd-d354-4a3e-a6ac-c91c9efcd31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:41.963155 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e1cd-d354-4a3e-a6ac-c91c9efcd31b admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:41.963614 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e1cd-d354-4a3e-a6ac-c91c9efcd31b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:41.963993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e1cd-d354-4a3e-a6ac-c91c9efcd31b admin admin] PATH_INFO: `/v3/projects/27e3379e9d764f23a3cd33a8af6c7dac/users/2bd61feb0bbf4022b2e292be10ba3dea/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:41.966182 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9e1cd-d354-4a3e-a6ac-c91c9efcd31b admin admin] RBAC: Authorizing `identity:create_grant(project_id=27e3379e9d764f23a3cd33a8af6c7dac, user_id=2bd61feb0bbf4022b2e292be10ba3dea, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:41.969095 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9e1cd-d354-4a3e-a6ac-c91c9efcd31b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:41.982906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 987/1962] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:41 2022] PUT /identity/v3/projects/27e3379e9d764f23a3cd33a8af6c7dac/users/2bd61feb0bbf4022b2e292be10ba3dea/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:41.984401 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e1cd-d354-4a3e-a6ac-c91c9efcd31b admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:41.995259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 976/1963] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:41 2022] POST /identity/v3/users => generated 288 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:41.999927 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdd119-b642-47fe-8ed9-966e90c5d276 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:42.000873 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdd119-b642-47fe-8ed9-966e90c5d276 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:42.002554 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdd119-b642-47fe-8ed9-966e90c5d276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:42.058548 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-27bdd119-b642-47fe-8ed9-966e90c5d276 None None] MFA Rules not processed for user `2bd61feb0bbf4022b2e292be10ba3dea`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:42.059827 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-27bdd119-b642-47fe-8ed9-966e90c5d276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:42.091265 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e1cd-d354-4a3e-a6ac-c91c9efcd31b admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:42.097641 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462dbeb3-c932-492e-91fd-0193384f20ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:42.103504 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462dbeb3-c932-492e-91fd-0193384f20ef admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:42.103760 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462dbeb3-c932-492e-91fd-0193384f20ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:42.104160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462dbeb3-c932-492e-91fd-0193384f20ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:42.104973 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462dbeb3-c932-492e-91fd-0193384f20ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:42.107418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462dbeb3-c932-492e-91fd-0193384f20ef admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:42.164957 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 977/1964] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:42.164957 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bdd119-b642-47fe-8ed9-966e90c5d276 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:42.182008 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bdd119-b642-47fe-8ed9-966e90c5d276 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:42.187158 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fa63b-52f9-48b6-a565-ae7cd14082fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:42.187477 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fa63b-52f9-48b6-a565-ae7cd14082fe admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:42.187755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fa63b-52f9-48b6-a565-ae7cd14082fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:42.188035 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fa63b-52f9-48b6-a565-ae7cd14082fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:42.188893 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fa63b-52f9-48b6-a565-ae7cd14082fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:42.190694 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fa63b-52f9-48b6-a565-ae7cd14082fe admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:42.202036 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 978/1965] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:42.204081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fa63b-52f9-48b6-a565-ae7cd14082fe admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:42.206497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f96fa63b-52f9-48b6-a565-ae7cd14082fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:42.222104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:42.233156 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 988/1966] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:41 2022] POST /identity/v3/projects => generated 383 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:42.234194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462dbeb3-c932-492e-91fd-0193384f20ef admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:42.308877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fa63b-52f9-48b6-a565-ae7cd14082fe admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:42.325631 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462dbeb3-c932-492e-91fd-0193384f20ef admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:42.341206 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b7241-3416-427f-be54-4eab6c2268e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:42.341907 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b7241-3416-427f-be54-4eab6c2268e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:42.342193 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b7241-3416-427f-be54-4eab6c2268e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:42.342468 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b7241-3416-427f-be54-4eab6c2268e1 admin admin] PATH_INFO: `/v3/projects/e116ee1c0bb24d7a85416cf054609717/users/08fc1d07f3a5456da7e168f30293acf4/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:42.365662 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:42.368246 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b7241-3416-427f-be54-4eab6c2268e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e116ee1c0bb24d7a85416cf054609717, user_id=08fc1d07f3a5456da7e168f30293acf4, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:42.371092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b7241-3416-427f-be54-4eab6c2268e1 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:42.376323 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67692fed-a5fb-4549-9bbf-5e8b3454ec4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:42.376981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67692fed-a5fb-4549-9bbf-5e8b3454ec4b service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:42.377370 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67692fed-a5fb-4549-9bbf-5e8b3454ec4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:42.377650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67692fed-a5fb-4549-9bbf-5e8b3454ec4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:42.451680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67692fed-a5fb-4549-9bbf-5e8b3454ec4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:42.453552 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67692fed-a5fb-4549-9bbf-5e8b3454ec4b service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:42.465033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 979/1967] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:42.466392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67692fed-a5fb-4549-9bbf-5e8b3454ec4b service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:42.478730 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67692fed-a5fb-4549-9bbf-5e8b3454ec4b service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:42.484186 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091648fe-4d25-4fbb-b86f-f74a0b427744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:42.484937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091648fe-4d25-4fbb-b86f-f74a0b427744 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:42.485295 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091648fe-4d25-4fbb-b86f-f74a0b427744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:42.485650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091648fe-4d25-4fbb-b86f-f74a0b427744 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:42.486544 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091648fe-4d25-4fbb-b86f-f74a0b427744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:42.489159 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091648fe-4d25-4fbb-b86f-f74a0b427744 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:42.493952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 989/1968] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:42 2022] PUT /identity/v3/projects/e116ee1c0bb24d7a85416cf054609717/users/08fc1d07f3a5456da7e168f30293acf4/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:42.498738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b7241-3416-427f-be54-4eab6c2268e1 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:42.575359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b7241-3416-427f-be54-4eab6c2268e1 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:42.580866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb464e-2723-4dfe-8082-1fb9c9ad6170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:42.581648 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb464e-2723-4dfe-8082-1fb9c9ad6170 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:42.582531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb464e-2723-4dfe-8082-1fb9c9ad6170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:42.582854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb464e-2723-4dfe-8082-1fb9c9ad6170 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:42.583696 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb464e-2723-4dfe-8082-1fb9c9ad6170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:42.585421 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb464e-2723-4dfe-8082-1fb9c9ad6170 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:42.587212 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 980/1969] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:42 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:42.587817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091648fe-4d25-4fbb-b86f-f74a0b427744 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:42.601294 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 990/1970] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:42.601495 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091648fe-4d25-4fbb-b86f-f74a0b427744 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:42.602661 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb464e-2723-4dfe-8082-1fb9c9ad6170 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:42.606505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbb4c4-a899-4c41-9ba0-b43455c9e299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:42.607430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbb4c4-a899-4c41-9ba0-b43455c9e299 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:42.608770 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbb4c4-a899-4c41-9ba0-b43455c9e299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:42.612352 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbb4c4-a899-4c41-9ba0-b43455c9e299 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:42.618241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbb4c4-a899-4c41-9ba0-b43455c9e299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:42.618241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbb4c4-a899-4c41-9ba0-b43455c9e299 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:42.618619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb464e-2723-4dfe-8082-1fb9c9ad6170 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:42.623853 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bad94-a96a-408d-a03e-8a7ad94bd4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:42.624351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bad94-a96a-408d-a03e-8a7ad94bd4c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:42.624573 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bad94-a96a-408d-a03e-8a7ad94bd4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:42.624966 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bad94-a96a-408d-a03e-8a7ad94bd4c3 admin admin] PATH_INFO: `/v3/projects/e116ee1c0bb24d7a85416cf054609717/users/08fc1d07f3a5456da7e168f30293acf4/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:42.628017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bad94-a96a-408d-a03e-8a7ad94bd4c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e116ee1c0bb24d7a85416cf054609717, user_id=08fc1d07f3a5456da7e168f30293acf4, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:42.630306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 981/1971] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:42.634407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbb4c4-a899-4c41-9ba0-b43455c9e299 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:42.635373 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bad94-a96a-408d-a03e-8a7ad94bd4c3 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:42.646822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbb4c4-a899-4c41-9ba0-b43455c9e299 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:42.648662 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 991/1972] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:42 2022] PUT /identity/v3/projects/e116ee1c0bb24d7a85416cf054609717/users/08fc1d07f3a5456da7e168f30293acf4/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:42.660252 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc760207-394d-4a88-901a-29abfb7dfe09 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:42.660685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc760207-394d-4a88-901a-29abfb7dfe09 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:42.661066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc760207-394d-4a88-901a-29abfb7dfe09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:42.697418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-dc760207-394d-4a88-901a-29abfb7dfe09 None None] MFA Rules not processed for user `08fc1d07f3a5456da7e168f30293acf4`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:42.699216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-dc760207-394d-4a88-901a-29abfb7dfe09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:42.749009 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40e10a-e559-4be1-9a42-63e88f321f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:42.750004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40e10a-e559-4be1-9a42-63e88f321f61 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:42.750448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40e10a-e559-4be1-9a42-63e88f321f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:42.750912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40e10a-e559-4be1-9a42-63e88f321f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:42.751705 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40e10a-e559-4be1-9a42-63e88f321f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:42.753660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40e10a-e559-4be1-9a42-63e88f321f61 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:42.766405 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 982/1973] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:26:42.774716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40e10a-e559-4be1-9a42-63e88f321f61 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:42.782659 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40e10a-e559-4be1-9a42-63e88f321f61 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:42.789674 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6bb5e-001a-4d99-8707-19c8b8ee4f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:42.803514 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6bb5e-001a-4d99-8707-19c8b8ee4f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:42.804419 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6bb5e-001a-4d99-8707-19c8b8ee4f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:42.804939 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6bb5e-001a-4d99-8707-19c8b8ee4f8f admin admin] PATH_INFO: `/v3/projects/a46041bcc9834d39a2f7726e3e4400c5/users/ff9751bb89c34880808ed9cda3cec47f/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:42.807138 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 992/1974] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:42.826491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6bb5e-001a-4d99-8707-19c8b8ee4f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a46041bcc9834d39a2f7726e3e4400c5, user_id=ff9751bb89c34880808ed9cda3cec47f, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:42.835283 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6bb5e-001a-4d99-8707-19c8b8ee4f8f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:42.842922 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc760207-394d-4a88-901a-29abfb7dfe09 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:42.913548 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:42.926431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 983/1975] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:42 2022] PUT /identity/v3/projects/a46041bcc9834d39a2f7726e3e4400c5/users/ff9751bb89c34880808ed9cda3cec47f/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:42.937488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc760207-394d-4a88-901a-29abfb7dfe09 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:42.938014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7005a40-cd98-46d4-a30c-1d499726196e None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:42.938190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7005a40-cd98-46d4-a30c-1d499726196e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:42.938463 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7005a40-cd98-46d4-a30c-1d499726196e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:42.978612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-a7005a40-cd98-46d4-a30c-1d499726196e None None] MFA Rules not processed for user `ff9751bb89c34880808ed9cda3cec47f`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:42.983043 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a7005a40-cd98-46d4-a30c-1d499726196e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:43.020449 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a4222-a464-4b71-b13f-f1f28261d5cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:43.021348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a4222-a464-4b71-b13f-f1f28261d5cd service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:43.021633 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a4222-a464-4b71-b13f-f1f28261d5cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:43.028552 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a4222-a464-4b71-b13f-f1f28261d5cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:43.104343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 984/1976] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:26:43.108759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7005a40-cd98-46d4-a30c-1d499726196e None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:43.112945 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a4222-a464-4b71-b13f-f1f28261d5cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:43.117184 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a4222-a464-4b71-b13f-f1f28261d5cd service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:43.126639 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 993/1977] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:43.243932 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7005a40-cd98-46d4-a30c-1d499726196e None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:43.253229 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c894540-6e7d-4bd5-b5e8-f0eedf2af603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:43.254758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c894540-6e7d-4bd5-b5e8-f0eedf2af603 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:43.255341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c894540-6e7d-4bd5-b5e8-f0eedf2af603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:43.256032 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c894540-6e7d-4bd5-b5e8-f0eedf2af603 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:43.256826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c894540-6e7d-4bd5-b5e8-f0eedf2af603 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:43.260541 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c894540-6e7d-4bd5-b5e8-f0eedf2af603 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:43.361113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 985/1978] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:43 2022] POST /identity/v3/projects => generated 383 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:26:43.366055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a4222-a464-4b71-b13f-f1f28261d5cd service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:43.383889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a4222-a464-4b71-b13f-f1f28261d5cd service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:43.488468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c894540-6e7d-4bd5-b5e8-f0eedf2af603 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:43.496879 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0f63c-fc57-4bc1-95ae-c50679b0f323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:43.499842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0f63c-fc57-4bc1-95ae-c50679b0f323 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:43.502358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0f63c-fc57-4bc1-95ae-c50679b0f323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:43.503033 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0f63c-fc57-4bc1-95ae-c50679b0f323 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:43.504981 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0f63c-fc57-4bc1-95ae-c50679b0f323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:43.509253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0f63c-fc57-4bc1-95ae-c50679b0f323 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:43.535618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:43.579240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c894540-6e7d-4bd5-b5e8-f0eedf2af603 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:43.584319 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40607faf-583f-44a8-9997-ae3f7d568795 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:43.585059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40607faf-583f-44a8-9997-ae3f7d568795 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:43.585354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40607faf-583f-44a8-9997-ae3f7d568795 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:43.585632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40607faf-583f-44a8-9997-ae3f7d568795 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:43.587580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-40607faf-583f-44a8-9997-ae3f7d568795 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:43.598863 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 994/1979] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:43 2022] POST /identity/v3/users => generated 298 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:26:43.603516 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0f63c-fc57-4bc1-95ae-c50679b0f323 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:43.615426 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0f63c-fc57-4bc1-95ae-c50679b0f323 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:43.622242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fc868-cb19-4b8e-be5c-d16e5ff5a5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:43.623546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fc868-cb19-4b8e-be5c-d16e5ff5a5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:43.623988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fc868-cb19-4b8e-be5c-d16e5ff5a5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:43.625306 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fc868-cb19-4b8e-be5c-d16e5ff5a5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:43.626188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9fc868-cb19-4b8e-be5c-d16e5ff5a5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:43.627973 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9fc868-cb19-4b8e-be5c-d16e5ff5a5a4 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:43.642678 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 995/1980] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:26:43.647080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fc868-cb19-4b8e-be5c-d16e5ff5a5a4 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:43.662492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fc868-cb19-4b8e-be5c-d16e5ff5a5a4 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:43.681279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8690a2-0cf6-4a7d-865b-a96f0d54dc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:43.681279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8690a2-0cf6-4a7d-865b-a96f0d54dc00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:43.681279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8690a2-0cf6-4a7d-865b-a96f0d54dc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:43.681279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8690a2-0cf6-4a7d-865b-a96f0d54dc00 admin admin] PATH_INFO: `/v3/projects/e91a36f8ad924915a570582c854896a9/users/9157437f1c0e4b1f9de3b3fad7ebdb3e/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:43.691778 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40607faf-583f-44a8-9997-ae3f7d568795 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:43.694050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40607faf-583f-44a8-9997-ae3f7d568795 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:43.696418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8690a2-0cf6-4a7d-865b-a96f0d54dc00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e91a36f8ad924915a570582c854896a9, user_id=9157437f1c0e4b1f9de3b3fad7ebdb3e, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:43.700114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8690a2-0cf6-4a7d-865b-a96f0d54dc00 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:43.720808 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 986/1981] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:43.788562 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 996/1982] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:43 2022] PUT /identity/v3/projects/e91a36f8ad924915a570582c854896a9/users/9157437f1c0e4b1f9de3b3fad7ebdb3e/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:43.792835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40607faf-583f-44a8-9997-ae3f7d568795 service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:43.805149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40607faf-583f-44a8-9997-ae3f7d568795 service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:43.900142 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5c222-d434-44c0-84e9-91186210b9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:43.900984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5c222-d434-44c0-84e9-91186210b9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:43.901265 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5c222-d434-44c0-84e9-91186210b9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:43.901568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5c222-d434-44c0-84e9-91186210b9c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:43.902214 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5c222-d434-44c0-84e9-91186210b9c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:43.903845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5c222-d434-44c0-84e9-91186210b9c7 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:43.919247 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 987/1983] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:43.921204 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8690a2-0cf6-4a7d-865b-a96f0d54dc00 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:43.933685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8690a2-0cf6-4a7d-865b-a96f0d54dc00 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:43.940559 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164499a-750b-421c-9532-4900dc8dd876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:43.942354 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164499a-750b-421c-9532-4900dc8dd876 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:43.943208 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164499a-750b-421c-9532-4900dc8dd876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:43.943491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164499a-750b-421c-9532-4900dc8dd876 admin admin] PATH_INFO: `/v3/projects/e91a36f8ad924915a570582c854896a9/users/9157437f1c0e4b1f9de3b3fad7ebdb3e/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:43.946180 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164499a-750b-421c-9532-4900dc8dd876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e91a36f8ad924915a570582c854896a9, user_id=9157437f1c0e4b1f9de3b3fad7ebdb3e, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:43.948885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164499a-750b-421c-9532-4900dc8dd876 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:43.963996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 997/1984] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:43 2022] PUT /identity/v3/projects/e91a36f8ad924915a570582c854896a9/users/9157437f1c0e4b1f9de3b3fad7ebdb3e/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:43.975424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f814a-ed99-4425-9fee-f1f06238ed9d None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:43.975713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f814a-ed99-4425-9fee-f1f06238ed9d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:43.976240 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f814a-ed99-4425-9fee-f1f06238ed9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:44.020081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-e71f814a-ed99-4425-9fee-f1f06238ed9d None None] MFA Rules not processed for user `9157437f1c0e4b1f9de3b3fad7ebdb3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:44.021692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e71f814a-ed99-4425-9fee-f1f06238ed9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:44.133819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 988/1985] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:44.280039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164499a-750b-421c-9532-4900dc8dd876 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:44.371539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164499a-750b-421c-9532-4900dc8dd876 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:44.376999 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668aaa4-955c-4ec6-b562-36bbbbf613f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:44.377533 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668aaa4-955c-4ec6-b562-36bbbbf613f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:44.377816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668aaa4-955c-4ec6-b562-36bbbbf613f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:44.378092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668aaa4-955c-4ec6-b562-36bbbbf613f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:44.389421 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668aaa4-955c-4ec6-b562-36bbbbf613f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:44.391364 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668aaa4-955c-4ec6-b562-36bbbbf613f8 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:44.401571 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 998/1986] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:44.501237 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:44.586299 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f814a-ed99-4425-9fee-f1f06238ed9d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:44.588137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e71f814a-ed99-4425-9fee-f1f06238ed9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:44.599482 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:44.612922 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f814a-ed99-4425-9fee-f1f06238ed9d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:44.618495 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8dfc48-f1bb-4037-80ad-7ba5f99475d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:44.619417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dfc48-f1bb-4037-80ad-7ba5f99475d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:44.620003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dfc48-f1bb-4037-80ad-7ba5f99475d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:44.620432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dfc48-f1bb-4037-80ad-7ba5f99475d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:44.691702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8dfc48-f1bb-4037-80ad-7ba5f99475d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:44.693849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8dfc48-f1bb-4037-80ad-7ba5f99475d3 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:44.702658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 989/1987] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:44.832453 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668aaa4-955c-4ec6-b562-36bbbbf613f8 service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:44.900142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668aaa4-955c-4ec6-b562-36bbbbf613f8 service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:44.904774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cd180-6b34-4c00-a094-b0ad8ac50016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:44.905431 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cd180-6b34-4c00-a094-b0ad8ac50016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:44.905712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cd180-6b34-4c00-a094-b0ad8ac50016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:44.905986 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cd180-6b34-4c00-a094-b0ad8ac50016 admin admin] PATH_INFO: `/v3/users/ccae720662b64207b9982808334ceb4c` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:44.907145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5cd180-6b34-4c00-a094-b0ad8ac50016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccae720662b64207b9982808334ceb4c)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:44.909082 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5cd180-6b34-4c00-a094-b0ad8ac50016 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:45.121836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1f5cd180-6b34-4c00-a094-b0ad8ac50016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccae720662b64207b9982808334ceb4c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:45.135598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1f5cd180-6b34-4c00-a094-b0ad8ac50016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccae720662b64207b9982808334ceb4c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:45.169400 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:45.214904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1f5cd180-6b34-4c00-a094-b0ad8ac50016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccae720662b64207b9982808334ceb4c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:45.222450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1f5cd180-6b34-4c00-a094-b0ad8ac50016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccae720662b64207b9982808334ceb4c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:45.223260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1f5cd180-6b34-4c00-a094-b0ad8ac50016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccae720662b64207b9982808334ceb4c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:45.230445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 999/1988] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:44 2022] DELETE /identity/v3/users/ccae720662b64207b9982808334ceb4c => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:45.235360 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8607b-7be1-41b0-9bf8-aa22938746e8 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:45.236485 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8607b-7be1-41b0-9bf8-aa22938746e8 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:45.236796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8607b-7be1-41b0-9bf8-aa22938746e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:45.257445 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-e6f8607b-7be1-41b0-9bf8-aa22938746e8 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:45.258700 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e6f8607b-7be1-41b0-9bf8-aa22938746e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:45.265316 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:45.321232 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 990/1989] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:45.330848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cd180-6b34-4c00-a094-b0ad8ac50016 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:45.332788 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-1f5cd180-6b34-4c00-a094-b0ad8ac50016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:45.396114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cd180-6b34-4c00-a094-b0ad8ac50016 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:45.399792 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eed593-c64b-4f83-9ad4-b941f9a358d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:45.400402 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eed593-c64b-4f83-9ad4-b941f9a358d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:45.400683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eed593-c64b-4f83-9ad4-b941f9a358d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:45.400958 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eed593-c64b-4f83-9ad4-b941f9a358d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:45.411382 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eed593-c64b-4f83-9ad4-b941f9a358d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:45.412688 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eed593-c64b-4f83-9ad4-b941f9a358d6 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:45.432788 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1000/1990] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:45.586788 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:45.637954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:45.992651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:46.060742 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:46.227797 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8607b-7be1-41b0-9bf8-aa22938746e8 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:46.229789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e6f8607b-7be1-41b0-9bf8-aa22938746e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:46.244727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8607b-7be1-41b0-9bf8-aa22938746e8 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:46.250981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:46.251773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:46.252881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:46.252881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4 admin admin] PATH_INFO: `/v3/projects/506f0d9adb804e79885908fd3a0c8665` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:46.256606 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=506f0d9adb804e79885908fd3a0c8665)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:46.258633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:46.304595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:46.424151 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '506f0d9adb804e79885908fd3a0c8665', 'request_id': 'req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:46.437255 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '506f0d9adb804e79885908fd3a0c8665', 'request_id': 'req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:46.440176 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:46.448920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '506f0d9adb804e79885908fd3a0c8665', 'request_id': 'req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:46.529222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4 admin admin] The token cache is being invalidate because project 506f0d9adb804e79885908fd3a0c8665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:46.529511 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:46.530135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:46.531727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 991/1991] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:46 2022] DELETE /identity/v3/projects/506f0d9adb804e79885908fd3a0c8665 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:46.536102 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eed593-c64b-4f83-9ad4-b941f9a358d6 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:46.545979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eed593-c64b-4f83-9ad4-b941f9a358d6 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:46.614677 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3afcba-24ff-473d-8722-0dd33166a325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:46.615354 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3afcba-24ff-473d-8722-0dd33166a325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:46.615639 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3afcba-24ff-473d-8722-0dd33166a325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:46.615916 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3afcba-24ff-473d-8722-0dd33166a325 admin admin] PATH_INFO: `/v3/users/2bd61feb0bbf4022b2e292be10ba3dea` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:46.617100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3afcba-24ff-473d-8722-0dd33166a325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bd61feb0bbf4022b2e292be10ba3dea)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:46.619114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3afcba-24ff-473d-8722-0dd33166a325 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:46.808079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2a3afcba-24ff-473d-8722-0dd33166a325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bd61feb0bbf4022b2e292be10ba3dea'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:46.821684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2a3afcba-24ff-473d-8722-0dd33166a325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bd61feb0bbf4022b2e292be10ba3dea'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:46.849961 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:46.900828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2a3afcba-24ff-473d-8722-0dd33166a325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bd61feb0bbf4022b2e292be10ba3dea'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:46.908308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2a3afcba-24ff-473d-8722-0dd33166a325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bd61feb0bbf4022b2e292be10ba3dea'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:46.909012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2a3afcba-24ff-473d-8722-0dd33166a325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bd61feb0bbf4022b2e292be10ba3dea'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:46.915474 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1001/1992] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:46 2022] DELETE /identity/v3/users/2bd61feb0bbf4022b2e292be10ba3dea => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:47.330297 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:47.331920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:47.345921 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7c873-f951-45de-ae7e-c49cc4ebffc4 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:47.410785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04f797-ddc0-4859-abb3-837ce2974340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:47.411538 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04f797-ddc0-4859-abb3-837ce2974340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:47.411810 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04f797-ddc0-4859-abb3-837ce2974340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:47.412085 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04f797-ddc0-4859-abb3-837ce2974340 admin admin] PATH_INFO: `/v3/projects/27e3379e9d764f23a3cd33a8af6c7dac` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:47.414417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04f797-ddc0-4859-abb3-837ce2974340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27e3379e9d764f23a3cd33a8af6c7dac)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:47.416481 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04f797-ddc0-4859-abb3-837ce2974340 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:47.523754 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-dd04f797-ddc0-4859-abb3-837ce2974340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27e3379e9d764f23a3cd33a8af6c7dac', 'request_id': 'req-dd04f797-ddc0-4859-abb3-837ce2974340'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:47.529789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-dd04f797-ddc0-4859-abb3-837ce2974340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27e3379e9d764f23a3cd33a8af6c7dac', 'request_id': 'req-dd04f797-ddc0-4859-abb3-837ce2974340'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:47.537013 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-dd04f797-ddc0-4859-abb3-837ce2974340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27e3379e9d764f23a3cd33a8af6c7dac', 'request_id': 'req-dd04f797-ddc0-4859-abb3-837ce2974340'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:47.643121 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-dd04f797-ddc0-4859-abb3-837ce2974340 admin admin] The token cache is being invalidate because project 27e3379e9d764f23a3cd33a8af6c7dac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:47.645841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-dd04f797-ddc0-4859-abb3-837ce2974340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:47.645841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-dd04f797-ddc0-4859-abb3-837ce2974340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:47.646323 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 992/1993] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:47 2022] DELETE /identity/v3/projects/27e3379e9d764f23a3cd33a8af6c7dac => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:47.652795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ea7bd-3afc-422e-a445-d5cd8edba7ce None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:47.653142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ea7bd-3afc-422e-a445-d5cd8edba7ce None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:47.653424 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ea7bd-3afc-422e-a445-d5cd8edba7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:47.679705 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-670ea7bd-3afc-422e-a445-d5cd8edba7ce None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:47.681129 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-670ea7bd-3afc-422e-a445-d5cd8edba7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:47.746902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1002/1994] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:47.754777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21647bdb-837a-45da-a1d8-a04cbb225f96 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:47.755225 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21647bdb-837a-45da-a1d8-a04cbb225f96 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:47.755565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21647bdb-837a-45da-a1d8-a04cbb225f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:47.781887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-21647bdb-837a-45da-a1d8-a04cbb225f96 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:47.783168 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-21647bdb-837a-45da-a1d8-a04cbb225f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:47.788433 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ea7bd-3afc-422e-a445-d5cd8edba7ce None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:47.789813 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-670ea7bd-3afc-422e-a445-d5cd8edba7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:47.794322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 993/1995] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:47.799317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21647bdb-837a-45da-a1d8-a04cbb225f96 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:47.805560 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ea7bd-3afc-422e-a445-d5cd8edba7ce None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:47.811221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21647bdb-837a-45da-a1d8-a04cbb225f96 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:47.812352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503a79c-e4c7-4d7c-9d59-b56eb55cfdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:47.813947 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503a79c-e4c7-4d7c-9d59-b56eb55cfdfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:47.814410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503a79c-e4c7-4d7c-9d59-b56eb55cfdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:47.814861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503a79c-e4c7-4d7c-9d59-b56eb55cfdfe admin admin] PATH_INFO: `/v3/users/988a3046b0f842b3b4dce1891217b0f6` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:47.816253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503a79c-e4c7-4d7c-9d59-b56eb55cfdfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=988a3046b0f842b3b4dce1891217b0f6)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:47.817151 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc99deb-4bc3-450e-aa2f-177aaed2130f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:47.817919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc99deb-4bc3-450e-aa2f-177aaed2130f admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:47.818267 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc99deb-4bc3-450e-aa2f-177aaed2130f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:47.819153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc99deb-4bc3-450e-aa2f-177aaed2130f admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:47.819880 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc99deb-4bc3-450e-aa2f-177aaed2130f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:47.820361 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc99deb-4bc3-450e-aa2f-177aaed2130f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:47.820959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503a79c-e4c7-4d7c-9d59-b56eb55cfdfe admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:47.822258 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc99deb-4bc3-450e-aa2f-177aaed2130f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:47.840804 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 994/1996] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:47.842323 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc99deb-4bc3-450e-aa2f-177aaed2130f admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:47.853697 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc99deb-4bc3-450e-aa2f-177aaed2130f admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:47.860018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05f52b-9947-4555-8395-12d3acb4cf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:47.860640 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05f52b-9947-4555-8395-12d3acb4cf79 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:47.860920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05f52b-9947-4555-8395-12d3acb4cf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:47.861188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05f52b-9947-4555-8395-12d3acb4cf79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:47.861820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05f52b-9947-4555-8395-12d3acb4cf79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:47.863777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05f52b-9947-4555-8395-12d3acb4cf79 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:47.949993 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 995/1997] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:47 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:26:47.954014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05f52b-9947-4555-8395-12d3acb4cf79 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:47.998401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:48.018886 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6503a79c-e4c7-4d7c-9d59-b56eb55cfdfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '988a3046b0f842b3b4dce1891217b0f6'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.024542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05f52b-9947-4555-8395-12d3acb4cf79 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:48.041810 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9352cc-6c93-44e3-a943-9dcab3bc68d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:48.041810 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9352cc-6c93-44e3-a943-9dcab3bc68d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:48.041810 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9352cc-6c93-44e3-a943-9dcab3bc68d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:48.041810 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9352cc-6c93-44e3-a943-9dcab3bc68d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:48.042187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9352cc-6c93-44e3-a943-9dcab3bc68d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:48.042187 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9352cc-6c93-44e3-a943-9dcab3bc68d5 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:48.045959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6503a79c-e4c7-4d7c-9d59-b56eb55cfdfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '988a3046b0f842b3b4dce1891217b0f6'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.145789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 996/1998] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:47 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:26:48.149147 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9352cc-6c93-44e3-a943-9dcab3bc68d5 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:48.158943 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9352cc-6c93-44e3-a943-9dcab3bc68d5 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:48.163888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8660fe-53f9-46f3-b519-940492672414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:48.164501 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8660fe-53f9-46f3-b519-940492672414 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:48.164780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8660fe-53f9-46f3-b519-940492672414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:48.165047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8660fe-53f9-46f3-b519-940492672414 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:48.165681 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8660fe-53f9-46f3-b519-940492672414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:48.167323 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8660fe-53f9-46f3-b519-940492672414 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:48.174839 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6503a79c-e4c7-4d7c-9d59-b56eb55cfdfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '988a3046b0f842b3b4dce1891217b0f6'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.178739 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 997/1999] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:26:48.182665 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8660fe-53f9-46f3-b519-940492672414 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:48.183636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6503a79c-e4c7-4d7c-9d59-b56eb55cfdfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '988a3046b0f842b3b4dce1891217b0f6'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.184432 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6503a79c-e4c7-4d7c-9d59-b56eb55cfdfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '988a3046b0f842b3b4dce1891217b0f6'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.190678 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1003/2000] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:47 2022] DELETE /identity/v3/users/988a3046b0f842b3b4dce1891217b0f6 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:48.192661 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8660fe-53f9-46f3-b519-940492672414 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:48.196546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3331af1-f225-4302-9fa4-cabb8a0a2640 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:48.197099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3331af1-f225-4302-9fa4-cabb8a0a2640 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:48.197447 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3331af1-f225-4302-9fa4-cabb8a0a2640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:48.205901 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:48.217232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a3331af1-f225-4302-9fa4-cabb8a0a2640 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:48.218427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a3331af1-f225-4302-9fa4-cabb8a0a2640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:48.250728 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a15f21-a69b-4a5c-af6a-5b50f0ea37a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:48.251556 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a15f21-a69b-4a5c-af6a-5b50f0ea37a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:48.252183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a15f21-a69b-4a5c-af6a-5b50f0ea37a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:48.252472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a15f21-a69b-4a5c-af6a-5b50f0ea37a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:48.253178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a15f21-a69b-4a5c-af6a-5b50f0ea37a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:48.255019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a15f21-a69b-4a5c-af6a-5b50f0ea37a4 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:48.270371 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 998/2001] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:26:48.279154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a15f21-a69b-4a5c-af6a-5b50f0ea37a4 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:48.281814 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-68a15f21-a69b-4a5c-af6a-5b50f0ea37a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:48.292016 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1004/2002] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:48.301376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3331af1-f225-4302-9fa4-cabb8a0a2640 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:48.302749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a3331af1-f225-4302-9fa4-cabb8a0a2640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:48.303427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a15f21-a69b-4a5c-af6a-5b50f0ea37a4 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:48.308303 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba1cdf-1409-4c4e-9d58-6a003efb5481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:48.308977 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba1cdf-1409-4c4e-9d58-6a003efb5481 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:48.309260 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba1cdf-1409-4c4e-9d58-6a003efb5481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:48.309543 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba1cdf-1409-4c4e-9d58-6a003efb5481 admin admin] PATH_INFO: `/v3/projects/b9112c2dba294780a160b156a0576924/users/d8ba1d1dd15344bb965260db0d3800cb/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:48.325515 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba1cdf-1409-4c4e-9d58-6a003efb5481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9112c2dba294780a160b156a0576924, user_id=d8ba1d1dd15344bb965260db0d3800cb, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:48.328251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba1cdf-1409-4c4e-9d58-6a003efb5481 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:48.362679 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3331af1-f225-4302-9fa4-cabb8a0a2640 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:48.367084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6533458-137b-4990-94aa-f8a3889e181b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:48.367691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6533458-137b-4990-94aa-f8a3889e181b service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:48.367969 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6533458-137b-4990-94aa-f8a3889e181b service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:48.368241 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6533458-137b-4990-94aa-f8a3889e181b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:48.377111 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6533458-137b-4990-94aa-f8a3889e181b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:48.378529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6533458-137b-4990-94aa-f8a3889e181b service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:48.387876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1005/2003] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:48.450246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 999/2004] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:48 2022] PUT /identity/v3/projects/b9112c2dba294780a160b156a0576924/users/d8ba1d1dd15344bb965260db0d3800cb/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:48.460956 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c31240-5aec-4c3d-9a2d-08e6b9b73760 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:48.461344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c31240-5aec-4c3d-9a2d-08e6b9b73760 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:48.461644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c31240-5aec-4c3d-9a2d-08e6b9b73760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:48.495356 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a0c31240-5aec-4c3d-9a2d-08e6b9b73760 None None] MFA Rules not processed for user `d8ba1d1dd15344bb965260db0d3800cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:48.496583 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a0c31240-5aec-4c3d-9a2d-08e6b9b73760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:48.571954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1006/2005] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:48.585445 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba1cdf-1409-4c4e-9d58-6a003efb5481 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:48.588531 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-73ba1cdf-1409-4c4e-9d58-6a003efb5481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:48.665447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba1cdf-1409-4c4e-9d58-6a003efb5481 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:48.674141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885c9c1-5863-4df5-8f0f-8b042a9e812a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:48.675181 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885c9c1-5863-4df5-8f0f-8b042a9e812a service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:48.675837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885c9c1-5863-4df5-8f0f-8b042a9e812a service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:48.676342 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885c9c1-5863-4df5-8f0f-8b042a9e812a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:48.687932 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885c9c1-5863-4df5-8f0f-8b042a9e812a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:48.689632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885c9c1-5863-4df5-8f0f-8b042a9e812a service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:48.700149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1000/2006] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:48.727019 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c31240-5aec-4c3d-9a2d-08e6b9b73760 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:48.770536 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:48.792628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c31240-5aec-4c3d-9a2d-08e6b9b73760 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:48.797322 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccff9a3-1247-4355-b5c9-fcd3d9e00218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:48.797954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccff9a3-1247-4355-b5c9-fcd3d9e00218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:48.798234 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccff9a3-1247-4355-b5c9-fcd3d9e00218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:48.798506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccff9a3-1247-4355-b5c9-fcd3d9e00218 admin admin] PATH_INFO: `/v3/users/d8ba1d1dd15344bb965260db0d3800cb` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:48.799675 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccff9a3-1247-4355-b5c9-fcd3d9e00218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8ba1d1dd15344bb965260db0d3800cb)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:48.803087 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccff9a3-1247-4355-b5c9-fcd3d9e00218 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:48.857905 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885c9c1-5863-4df5-8f0f-8b042a9e812a service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:48.859418 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-3885c9c1-5863-4df5-8f0f-8b042a9e812a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:48.883452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885c9c1-5863-4df5-8f0f-8b042a9e812a service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:48.888678 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70a1a8-a918-4be9-a80e-e099764618d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:48.889334 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70a1a8-a918-4be9-a80e-e099764618d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:48.889630 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70a1a8-a918-4be9-a80e-e099764618d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:48.889930 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70a1a8-a918-4be9-a80e-e099764618d0 admin admin] PATH_INFO: `/v3/projects/1243a90f5170471fa3f752e6176bc078` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:48.891327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee70a1a8-a918-4be9-a80e-e099764618d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1243a90f5170471fa3f752e6176bc078)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:48.893354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee70a1a8-a918-4be9-a80e-e099764618d0 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:48.976420 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-dccff9a3-1247-4355-b5c9-fcd3d9e00218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8ba1d1dd15344bb965260db0d3800cb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.986630 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:48.988444 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-dccff9a3-1247-4355-b5c9-fcd3d9e00218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8ba1d1dd15344bb965260db0d3800cb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.004116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ee70a1a8-a918-4be9-a80e-e099764618d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1243a90f5170471fa3f752e6176bc078', 'request_id': 'req-ee70a1a8-a918-4be9-a80e-e099764618d0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.009661 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ee70a1a8-a918-4be9-a80e-e099764618d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1243a90f5170471fa3f752e6176bc078', 'request_id': 'req-ee70a1a8-a918-4be9-a80e-e099764618d0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.016037 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ee70a1a8-a918-4be9-a80e-e099764618d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1243a90f5170471fa3f752e6176bc078', 'request_id': 'req-ee70a1a8-a918-4be9-a80e-e099764618d0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.087519 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-dccff9a3-1247-4355-b5c9-fcd3d9e00218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8ba1d1dd15344bb965260db0d3800cb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.095088 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-dccff9a3-1247-4355-b5c9-fcd3d9e00218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8ba1d1dd15344bb965260db0d3800cb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.097647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-dccff9a3-1247-4355-b5c9-fcd3d9e00218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8ba1d1dd15344bb965260db0d3800cb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.102499 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1007/2007] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:48 2022] DELETE /identity/v3/users/d8ba1d1dd15344bb965260db0d3800cb => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:49.105858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccff9a3-1247-4355-b5c9-fcd3d9e00218 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:49.106273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-dccff9a3-1247-4355-b5c9-fcd3d9e00218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:49.157809 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ee70a1a8-a918-4be9-a80e-e099764618d0 admin admin] The token cache is being invalidate because project 1243a90f5170471fa3f752e6176bc078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:49.158179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ee70a1a8-a918-4be9-a80e-e099764618d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.158784 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ee70a1a8-a918-4be9-a80e-e099764618d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.160546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1001/2008] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:48 2022] DELETE /identity/v3/projects/1243a90f5170471fa3f752e6176bc078 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:49.161837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79994bfc-eebc-41e2-b57e-f827c7e6169b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:49.162150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79994bfc-eebc-41e2-b57e-f827c7e6169b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:49.162572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79994bfc-eebc-41e2-b57e-f827c7e6169b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:49.179067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccff9a3-1247-4355-b5c9-fcd3d9e00218 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:49.182654 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-79994bfc-eebc-41e2-b57e-f827c7e6169b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:49.184018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-79994bfc-eebc-41e2-b57e-f827c7e6169b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:49.184132 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4494732-94cd-4a31-b269-77d3fbcbd065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:49.184726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4494732-94cd-4a31-b269-77d3fbcbd065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:49.185010 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4494732-94cd-4a31-b269-77d3fbcbd065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:49.185296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4494732-94cd-4a31-b269-77d3fbcbd065 admin admin] PATH_INFO: `/v3/users/ff9751bb89c34880808ed9cda3cec47f` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:49.186396 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4494732-94cd-4a31-b269-77d3fbcbd065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff9751bb89c34880808ed9cda3cec47f)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:49.188356 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4494732-94cd-4a31-b269-77d3fbcbd065 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:49.192312 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1002/2009] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:49.196260 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79994bfc-eebc-41e2-b57e-f827c7e6169b None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:49.196260 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-79994bfc-eebc-41e2-b57e-f827c7e6169b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:49.207174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79994bfc-eebc-41e2-b57e-f827c7e6169b None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:49.213870 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007255fc-a3a0-4356-bbc0-f2fd1c36e1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:49.214555 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007255fc-a3a0-4356-bbc0-f2fd1c36e1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:49.214843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007255fc-a3a0-4356-bbc0-f2fd1c36e1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:49.218518 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007255fc-a3a0-4356-bbc0-f2fd1c36e1ba admin admin] PATH_INFO: `/v3/users/08fc1d07f3a5456da7e168f30293acf4` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:49.218518 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007255fc-a3a0-4356-bbc0-f2fd1c36e1ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=08fc1d07f3a5456da7e168f30293acf4)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:49.218708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007255fc-a3a0-4356-bbc0-f2fd1c36e1ba admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:49.424596 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c4494732-94cd-4a31-b269-77d3fbcbd065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff9751bb89c34880808ed9cda3cec47f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.433136 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:49.437739 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c4494732-94cd-4a31-b269-77d3fbcbd065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff9751bb89c34880808ed9cda3cec47f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.446645 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-007255fc-a3a0-4356-bbc0-f2fd1c36e1ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08fc1d07f3a5456da7e168f30293acf4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.546173 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c4494732-94cd-4a31-b269-77d3fbcbd065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff9751bb89c34880808ed9cda3cec47f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.554619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c4494732-94cd-4a31-b269-77d3fbcbd065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff9751bb89c34880808ed9cda3cec47f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.555305 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c4494732-94cd-4a31-b269-77d3fbcbd065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff9751bb89c34880808ed9cda3cec47f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.561946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1008/2010] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:49 2022] DELETE /identity/v3/users/ff9751bb89c34880808ed9cda3cec47f => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:49.562739 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4494732-94cd-4a31-b269-77d3fbcbd065 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:49.564117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c4494732-94cd-4a31-b269-77d3fbcbd065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:49.597872 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-007255fc-a3a0-4356-bbc0-f2fd1c36e1ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08fc1d07f3a5456da7e168f30293acf4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.598532 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-007255fc-a3a0-4356-bbc0-f2fd1c36e1ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08fc1d07f3a5456da7e168f30293acf4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.613402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-007255fc-a3a0-4356-bbc0-f2fd1c36e1ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08fc1d07f3a5456da7e168f30293acf4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.614022 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-007255fc-a3a0-4356-bbc0-f2fd1c36e1ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08fc1d07f3a5456da7e168f30293acf4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.625019 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4494732-94cd-4a31-b269-77d3fbcbd065 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:49.628909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc6ff4-b949-4b3b-b9ba-01166960c839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:49.629441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1003/2011] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:49 2022] DELETE /identity/v3/users/08fc1d07f3a5456da7e168f30293acf4 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:49.629568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc6ff4-b949-4b3b-b9ba-01166960c839 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:49.629804 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc6ff4-b949-4b3b-b9ba-01166960c839 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:49.630081 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc6ff4-b949-4b3b-b9ba-01166960c839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:49.630779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f32198-5ef3-4310-bbc4-ac93a20b2c46 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:49.631771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f32198-5ef3-4310-bbc4-ac93a20b2c46 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:49.632090 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f32198-5ef3-4310-bbc4-ac93a20b2c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:49.632756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-bdbc6ff4-b949-4b3b-b9ba-01166960c839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:49.652838 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-13f32198-5ef3-4310-bbc4-ac93a20b2c46 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:49.654786 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-13f32198-5ef3-4310-bbc4-ac93a20b2c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:49.711186 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc6ff4-b949-4b3b-b9ba-01166960c839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:49.715459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc6ff4-b949-4b3b-b9ba-01166960c839 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:49.744763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1009/2012] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:49.746525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1004/2013] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:49.757909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc6ff4-b949-4b3b-b9ba-01166960c839 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:49.759455 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-bdbc6ff4-b949-4b3b-b9ba-01166960c839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:49.776928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc6ff4-b949-4b3b-b9ba-01166960c839 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:49.840330 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1508186-9746-4a77-b61d-616bbe49ac55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:49.840917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1508186-9746-4a77-b61d-616bbe49ac55 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:49.841190 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1508186-9746-4a77-b61d-616bbe49ac55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:49.841649 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1508186-9746-4a77-b61d-616bbe49ac55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:49.853772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1508186-9746-4a77-b61d-616bbe49ac55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:49.855572 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1508186-9746-4a77-b61d-616bbe49ac55 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:49.867244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1010/2014] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:50.209305 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f32198-5ef3-4310-bbc4-ac93a20b2c46 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:50.210932 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-13f32198-5ef3-4310-bbc4-ac93a20b2c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:50.233697 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f32198-5ef3-4310-bbc4-ac93a20b2c46 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:50.236626 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1508186-9746-4a77-b61d-616bbe49ac55 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:50.239165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c1508186-9746-4a77-b61d-616bbe49ac55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:50.247345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:50.247685 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.247972 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.248618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91 admin admin] PATH_INFO: `/v3/projects/b9112c2dba294780a160b156a0576924` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.251291 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9112c2dba294780a160b156a0576924)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:50.253674 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1508186-9746-4a77-b61d-616bbe49ac55 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:50.254635 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:50.260566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdaf87c-e232-4a8b-83bf-245aa303964a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:50.261385 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdaf87c-e232-4a8b-83bf-245aa303964a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.261789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdaf87c-e232-4a8b-83bf-245aa303964a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.262075 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdaf87c-e232-4a8b-83bf-245aa303964a admin admin] PATH_INFO: `/v3/projects/e116ee1c0bb24d7a85416cf054609717` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.263390 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdaf87c-e232-4a8b-83bf-245aa303964a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e116ee1c0bb24d7a85416cf054609717)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:50.267166 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdaf87c-e232-4a8b-83bf-245aa303964a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:50.297022 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:50.310496 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:50.412214 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9112c2dba294780a160b156a0576924', 'request_id': 'req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.418124 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9112c2dba294780a160b156a0576924', 'request_id': 'req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.424809 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9112c2dba294780a160b156a0576924', 'request_id': 'req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.426617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-0bdaf87c-e232-4a8b-83bf-245aa303964a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e116ee1c0bb24d7a85416cf054609717', 'request_id': 'req-0bdaf87c-e232-4a8b-83bf-245aa303964a'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.433883 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-0bdaf87c-e232-4a8b-83bf-245aa303964a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e116ee1c0bb24d7a85416cf054609717', 'request_id': 'req-0bdaf87c-e232-4a8b-83bf-245aa303964a'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.507010 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91 admin admin] The token cache is being invalidate because project b9112c2dba294780a160b156a0576924 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:50.507274 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.507952 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.509467 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1005/2015] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:50 2022] DELETE /identity/v3/projects/b9112c2dba294780a160b156a0576924 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:50.512800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:50.512800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:50.569674 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-0bdaf87c-e232-4a8b-83bf-245aa303964a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e116ee1c0bb24d7a85416cf054609717', 'request_id': 'req-0bdaf87c-e232-4a8b-83bf-245aa303964a'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.578355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a24d4e-d086-4ba0-819e-dc8e3662ca91 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:50.584262 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:50.584758 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-0bdaf87c-e232-4a8b-83bf-245aa303964a admin admin] The token cache is being invalidate because project e116ee1c0bb24d7a85416cf054609717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:50.584976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.585246 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-0bdaf87c-e232-4a8b-83bf-245aa303964a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.585358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.585643 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1 admin admin] PATH_INFO: `/v3/projects/a46041bcc9834d39a2f7726e3e4400c5` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.586514 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-0bdaf87c-e232-4a8b-83bf-245aa303964a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.586777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a46041bcc9834d39a2f7726e3e4400c5)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:50.588083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1011/2016] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:50 2022] DELETE /identity/v3/projects/e116ee1c0bb24d7a85416cf054609717 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:50.588748 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:50.589277 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49db440-d552-440c-85a6-f8a50499a808 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.589560 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49db440-d552-440c-85a6-f8a50499a808 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.589833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49db440-d552-440c-85a6-f8a50499a808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.610817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a49db440-d552-440c-85a6-f8a50499a808 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:50.612143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a49db440-d552-440c-85a6-f8a50499a808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:50.622295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1012/2017] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:50.623643 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e8502-887d-4b3f-aa15-0c8a61f65b60 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.623931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e8502-887d-4b3f-aa15-0c8a61f65b60 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.624203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e8502-887d-4b3f-aa15-0c8a61f65b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.644598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-161e8502-887d-4b3f-aa15-0c8a61f65b60 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:50.645778 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-161e8502-887d-4b3f-aa15-0c8a61f65b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:50.654077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1013/2018] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:50.655289 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ac832-070e-4e7c-b58f-ae5013cb532a None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.655568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ac832-070e-4e7c-b58f-ae5013cb532a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.655839 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ac832-070e-4e7c-b58f-ae5013cb532a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.675418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-3d6ac832-070e-4e7c-b58f-ae5013cb532a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:50.676844 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3d6ac832-070e-4e7c-b58f-ae5013cb532a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:50.704325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a46041bcc9834d39a2f7726e3e4400c5', 'request_id': 'req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.710760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a46041bcc9834d39a2f7726e3e4400c5', 'request_id': 'req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.717156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a46041bcc9834d39a2f7726e3e4400c5', 'request_id': 'req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.742315 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1014/2019] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:50.743743 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6be26-11fd-4bd4-9a68-1be6dd4b294e None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.744055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6be26-11fd-4bd4-9a68-1be6dd4b294e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.744342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6be26-11fd-4bd4-9a68-1be6dd4b294e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.765549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-89d6be26-11fd-4bd4-9a68-1be6dd4b294e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:50.769478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-89d6be26-11fd-4bd4-9a68-1be6dd4b294e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:50.782744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1015/2020] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:50.783683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6be26-11fd-4bd4-9a68-1be6dd4b294e None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:50.788034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1 admin admin] The token cache is being invalidate because project a46041bcc9834d39a2f7726e3e4400c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:50.788265 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.788990 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.791414 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1006/2021] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:50 2022] DELETE /identity/v3/projects/a46041bcc9834d39a2f7726e3e4400c5 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:50.792176 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:50.799444 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6be26-11fd-4bd4-9a68-1be6dd4b294e None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:50.800502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:50.804499 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210292ae-5513-48a1-97a4-8f07a36ce390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:50.804754 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210292ae-5513-48a1-97a4-8f07a36ce390 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.805027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210292ae-5513-48a1-97a4-8f07a36ce390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.805302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210292ae-5513-48a1-97a4-8f07a36ce390 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.805955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210292ae-5513-48a1-97a4-8f07a36ce390 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:50.806263 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210292ae-5513-48a1-97a4-8f07a36ce390 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:50.807891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210292ae-5513-48a1-97a4-8f07a36ce390 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:50.816352 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf67aa-105b-4f95-94bd-f8b2ac211ce1 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:50.821581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8a0fc-fc37-418e-bf1a-99087bfb67ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:50.821969 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1016/2022] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:50.822257 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8a0fc-fc37-418e-bf1a-99087bfb67ed admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.822471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8a0fc-fc37-418e-bf1a-99087bfb67ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.822744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8a0fc-fc37-418e-bf1a-99087bfb67ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.823426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8a0fc-fc37-418e-bf1a-99087bfb67ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:50.823730 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8a0fc-fc37-418e-bf1a-99087bfb67ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:50.824032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210292ae-5513-48a1-97a4-8f07a36ce390 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:50.825886 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8a0fc-fc37-418e-bf1a-99087bfb67ed admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:50.826439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-210292ae-5513-48a1-97a4-8f07a36ce390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:50.841537 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1007/2023] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:50.841537 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8a0fc-fc37-418e-bf1a-99087bfb67ed admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:50.841808 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210292ae-5513-48a1-97a4-8f07a36ce390 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:50.841898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-79f8a0fc-fc37-418e-bf1a-99087bfb67ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:50.844583 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5f204-2c0f-42fa-b7df-7cc5ab7a63ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:50.845217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5f204-2c0f-42fa-b7df-7cc5ab7a63ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.845499 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5f204-2c0f-42fa-b7df-7cc5ab7a63ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.845795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5f204-2c0f-42fa-b7df-7cc5ab7a63ae admin admin] PATH_INFO: `/v3/users/9157437f1c0e4b1f9de3b3fad7ebdb3e` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.846987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5f204-2c0f-42fa-b7df-7cc5ab7a63ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=9157437f1c0e4b1f9de3b3fad7ebdb3e)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:50.848901 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5f204-2c0f-42fa-b7df-7cc5ab7a63ae admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:50.853986 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8a0fc-fc37-418e-bf1a-99087bfb67ed admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:50.858813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0afd24c-b849-4517-921f-e7649b6f9941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:50.859464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afd24c-b849-4517-921f-e7649b6f9941 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.859744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afd24c-b849-4517-921f-e7649b6f9941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.860011 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afd24c-b849-4517-921f-e7649b6f9941 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.860648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afd24c-b849-4517-921f-e7649b6f9941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:50.862570 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afd24c-b849-4517-921f-e7649b6f9941 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.017230 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:51.026590 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1008/2024] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:50 2022] POST /identity/v3/projects => generated 379 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:51.029758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0afd24c-b849-4517-921f-e7649b6f9941 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.096022 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0afd24c-b849-4517-921f-e7649b6f9941 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.101968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8487147-c785-4358-b641-9d75239fa201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.102718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8487147-c785-4358-b641-9d75239fa201 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.103055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8487147-c785-4358-b641-9d75239fa201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.103336 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8487147-c785-4358-b641-9d75239fa201 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.104821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8487147-c785-4358-b641-9d75239fa201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:51.106080 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8487147-c785-4358-b641-9d75239fa201 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.112019 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-26d5f204-2c0f-42fa-b7df-7cc5ab7a63ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9157437f1c0e4b1f9de3b3fad7ebdb3e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:51.125751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-26d5f204-2c0f-42fa-b7df-7cc5ab7a63ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9157437f1c0e4b1f9de3b3fad7ebdb3e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:51.188211 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1009/2025] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:51 2022] POST /identity/v3/projects => generated 409 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:51.189190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8487147-c785-4358-b641-9d75239fa201 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.209387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-26d5f204-2c0f-42fa-b7df-7cc5ab7a63ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9157437f1c0e4b1f9de3b3fad7ebdb3e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:51.211787 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:51.217444 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-26d5f204-2c0f-42fa-b7df-7cc5ab7a63ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9157437f1c0e4b1f9de3b3fad7ebdb3e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:51.218104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-26d5f204-2c0f-42fa-b7df-7cc5ab7a63ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9157437f1c0e4b1f9de3b3fad7ebdb3e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:51.224602 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1017/2026] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:50 2022] DELETE /identity/v3/users/9157437f1c0e4b1f9de3b3fad7ebdb3e => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:51.229205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5f204-2c0f-42fa-b7df-7cc5ab7a63ae admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.230617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-26d5f204-2c0f-42fa-b7df-7cc5ab7a63ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:51.266350 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8487147-c785-4358-b641-9d75239fa201 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.307630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5f204-2c0f-42fa-b7df-7cc5ab7a63ae admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.316776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c58d2-6411-4fdb-a075-901a37d51e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.316776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c58d2-6411-4fdb-a075-901a37d51e43 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.316776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c58d2-6411-4fdb-a075-901a37d51e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.316776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c58d2-6411-4fdb-a075-901a37d51e43 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.317256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c58d2-6411-4fdb-a075-901a37d51e43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:51.317256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c58d2-6411-4fdb-a075-901a37d51e43 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.351690 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411997c-a67a-4cc2-9705-a6a0935fcdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.352315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411997c-a67a-4cc2-9705-a6a0935fcdd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.352626 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411997c-a67a-4cc2-9705-a6a0935fcdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.352902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411997c-a67a-4cc2-9705-a6a0935fcdd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.353540 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411997c-a67a-4cc2-9705-a6a0935fcdd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:51.355503 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411997c-a67a-4cc2-9705-a6a0935fcdd3 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.416989 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1018/2027] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:51 2022] POST /identity/v3/users => generated 305 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:51.418084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c58d2-6411-4fdb-a075-901a37d51e43 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.428295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c58d2-6411-4fdb-a075-901a37d51e43 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.433106 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a10ea-52c7-414d-9d4c-7196feb71a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.433705 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a10ea-52c7-414d-9d4c-7196feb71a8b admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.433988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a10ea-52c7-414d-9d4c-7196feb71a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.434276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a10ea-52c7-414d-9d4c-7196feb71a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.435020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a10ea-52c7-414d-9d4c-7196feb71a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:51.436631 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a10ea-52c7-414d-9d4c-7196feb71a8b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.443113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1010/2028] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:51 2022] POST /identity/v3/users => generated 290 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:51.448217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1019/2029] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:51.448808 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411997c-a67a-4cc2-9705-a6a0935fcdd3 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.450771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-1411997c-a67a-4cc2-9705-a6a0935fcdd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:51.453032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a10ea-52c7-414d-9d4c-7196feb71a8b admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.469613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411997c-a67a-4cc2-9705-a6a0935fcdd3 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.473233 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a10ea-52c7-414d-9d4c-7196feb71a8b admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.476601 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5bcd8-23d8-46ea-93a1-07b4d7d38166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.477916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5bcd8-23d8-46ea-93a1-07b4d7d38166 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.478212 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5bcd8-23d8-46ea-93a1-07b4d7d38166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.478485 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5bcd8-23d8-46ea-93a1-07b4d7d38166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.479205 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5bcd8-23d8-46ea-93a1-07b4d7d38166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:51.481047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5bcd8-23d8-46ea-93a1-07b4d7d38166 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.485751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a959a34-6ad5-4649-b243-1386d8775ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.486502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a959a34-6ad5-4649-b243-1386d8775ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.487056 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a959a34-6ad5-4649-b243-1386d8775ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.487315 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a959a34-6ad5-4649-b243-1386d8775ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.488071 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a959a34-6ad5-4649-b243-1386d8775ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:51.489779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a959a34-6ad5-4649-b243-1386d8775ca3 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.494835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1011/2030] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:51.497159 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5bcd8-23d8-46ea-93a1-07b4d7d38166 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.505411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1020/2031] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:51.508612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5bcd8-23d8-46ea-93a1-07b4d7d38166 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.513884 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887f979-62aa-4a1a-883b-3d2dac396f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.514701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887f979-62aa-4a1a-883b-3d2dac396f2a admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.515019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887f979-62aa-4a1a-883b-3d2dac396f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.515299 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887f979-62aa-4a1a-883b-3d2dac396f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.515408 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a959a34-6ad5-4649-b243-1386d8775ca3 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.516056 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887f979-62aa-4a1a-883b-3d2dac396f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:51.517662 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887f979-62aa-4a1a-883b-3d2dac396f2a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.529974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1012/2032] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:26:51.532735 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a959a34-6ad5-4649-b243-1386d8775ca3 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.534216 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887f979-62aa-4a1a-883b-3d2dac396f2a admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.542859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7174c74-1205-4661-b0be-0b9d852742e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.542859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7174c74-1205-4661-b0be-0b9d852742e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.542859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7174c74-1205-4661-b0be-0b9d852742e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.542859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7174c74-1205-4661-b0be-0b9d852742e1 admin admin] PATH_INFO: `/v3/projects/e30b6e1da3f84b46b24305d175e41762/users/ad2d61e33ade485784a3cb304ac02918/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.546293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887f979-62aa-4a1a-883b-3d2dac396f2a admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.552055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e67c1-f744-45a9-bd8d-708b77fb5c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.553243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e67c1-f744-45a9-bd8d-708b77fb5c30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.553243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e67c1-f744-45a9-bd8d-708b77fb5c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.553432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e67c1-f744-45a9-bd8d-708b77fb5c30 admin admin] PATH_INFO: `/v3/projects/b8bfa72ff61547e5a3097e5b40483f8a/users/be55f0bb21f044b0a8346cf953b1650e/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.557641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:51.563619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7174c74-1205-4661-b0be-0b9d852742e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e30b6e1da3f84b46b24305d175e41762, user_id=ad2d61e33ade485784a3cb304ac02918, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:51.566825 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7174c74-1205-4661-b0be-0b9d852742e1 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.576952 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e67c1-f744-45a9-bd8d-708b77fb5c30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8bfa72ff61547e5a3097e5b40483f8a, user_id=be55f0bb21f044b0a8346cf953b1650e, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:51.579682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e67c1-f744-45a9-bd8d-708b77fb5c30 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.630481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1021/2033] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:51 2022] PUT /identity/v3/projects/e30b6e1da3f84b46b24305d175e41762/users/ad2d61e33ade485784a3cb304ac02918/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:51.641449 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995a8b0-7be8-4878-91ff-7bc6f502d0b0 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.641743 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995a8b0-7be8-4878-91ff-7bc6f502d0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.642021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995a8b0-7be8-4878-91ff-7bc6f502d0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.654245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1013/2034] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:51 2022] PUT /identity/v3/projects/b8bfa72ff61547e5a3097e5b40483f8a/users/be55f0bb21f044b0a8346cf953b1650e/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:51.660239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d43c8-30dd-47ea-9cea-f7d40225577e None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.660704 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d43c8-30dd-47ea-9cea-f7d40225577e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.661051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d43c8-30dd-47ea-9cea-f7d40225577e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.674021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-f995a8b0-7be8-4878-91ff-7bc6f502d0b0 None None] MFA Rules not processed for user `ad2d61e33ade485784a3cb304ac02918`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:51.675340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f995a8b0-7be8-4878-91ff-7bc6f502d0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:51.681378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-e60d43c8-30dd-47ea-9cea-f7d40225577e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:51.683588 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e60d43c8-30dd-47ea-9cea-f7d40225577e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:51.757937 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1022/2035] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:51 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:26:51.757937 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995a8b0-7be8-4878-91ff-7bc6f502d0b0 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.757937 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f995a8b0-7be8-4878-91ff-7bc6f502d0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:51.776955 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1014/2036] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:26:51.776955 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d43c8-30dd-47ea-9cea-f7d40225577e None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.777585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995a8b0-7be8-4878-91ff-7bc6f502d0b0 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.777678 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d43c8-30dd-47ea-9cea-f7d40225577e None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.777854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5947c14-70fb-49ed-aba3-538f168743b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.778515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5947c14-70fb-49ed-aba3-538f168743b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.778799 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5947c14-70fb-49ed-aba3-538f168743b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.779109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5947c14-70fb-49ed-aba3-538f168743b5 admin admin] PATH_INFO: `/v3/projects/e91a36f8ad924915a570582c854896a9` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.780224 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5947c14-70fb-49ed-aba3-538f168743b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e91a36f8ad924915a570582c854896a9)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:51.782140 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5947c14-70fb-49ed-aba3-538f168743b5 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.782352 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d19733-520b-4540-aa95-d1cca31d3aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.783048 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d19733-520b-4540-aa95-d1cca31d3aab admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.783554 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d19733-520b-4540-aa95-d1cca31d3aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.783877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d19733-520b-4540-aa95-d1cca31d3aab admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.784512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d19733-520b-4540-aa95-d1cca31d3aab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:51.786402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d19733-520b-4540-aa95-d1cca31d3aab admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.806932 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:51.888250 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1015/2037] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:51 2022] POST /identity/v3/projects => generated 409 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:51.888250 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d19733-520b-4540-aa95-d1cca31d3aab admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.888250 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c0d19733-520b-4540-aa95-d1cca31d3aab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:51.900256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f5947c14-70fb-49ed-aba3-538f168743b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e91a36f8ad924915a570582c854896a9', 'request_id': 'req-f5947c14-70fb-49ed-aba3-538f168743b5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:51.906095 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f5947c14-70fb-49ed-aba3-538f168743b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e91a36f8ad924915a570582c854896a9', 'request_id': 'req-f5947c14-70fb-49ed-aba3-538f168743b5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:51.948048 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d19733-520b-4540-aa95-d1cca31d3aab admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.951785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c31e19-cc7e-4e2c-a251-fc17069cbf8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.954146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c31e19-cc7e-4e2c-a251-fc17069cbf8f service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.954430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c31e19-cc7e-4e2c-a251-fc17069cbf8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.954711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c31e19-cc7e-4e2c-a251-fc17069cbf8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.973153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f5947c14-70fb-49ed-aba3-538f168743b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e91a36f8ad924915a570582c854896a9', 'request_id': 'req-f5947c14-70fb-49ed-aba3-538f168743b5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:51.985615 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f5947c14-70fb-49ed-aba3-538f168743b5 admin admin] The token cache is being invalidate because project e91a36f8ad924915a570582c854896a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:51.985943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f5947c14-70fb-49ed-aba3-538f168743b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:51.986616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f5947c14-70fb-49ed-aba3-538f168743b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:51.988217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1023/2038] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:51 2022] DELETE /identity/v3/projects/e91a36f8ad924915a570582c854896a9 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:51.988969 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5947c14-70fb-49ed-aba3-538f168743b5 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.991824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f5947c14-70fb-49ed-aba3-538f168743b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:52.020349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c31e19-cc7e-4e2c-a251-fc17069cbf8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:52.021952 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c31e19-cc7e-4e2c-a251-fc17069cbf8f service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:52.032429 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1016/2039] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:52.033899 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd81e1-99cc-46f0-88a2-20c7826ee87e None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.034170 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd81e1-99cc-46f0-88a2-20c7826ee87e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.034455 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd81e1-99cc-46f0-88a2-20c7826ee87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.051591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:52.067949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-84bd81e1-99cc-46f0-88a2-20c7826ee87e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:52.069464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-84bd81e1-99cc-46f0-88a2-20c7826ee87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:52.079200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1017/2040] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:52.087249 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf6644-efde-437e-8bf3-61b708f5e83c None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.087504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf6644-efde-437e-8bf3-61b708f5e83c None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.087832 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf6644-efde-437e-8bf3-61b708f5e83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.092800 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5947c14-70fb-49ed-aba3-538f168743b5 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:52.098006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0a5c5-2eea-43ce-8a9c-da8d9e565a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:52.098628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0a5c5-2eea-43ce-8a9c-da8d9e565a20 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.098950 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0a5c5-2eea-43ce-8a9c-da8d9e565a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.099243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0a5c5-2eea-43ce-8a9c-da8d9e565a20 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.100993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b0a5c5-2eea-43ce-8a9c-da8d9e565a20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:52.103746 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b0a5c5-2eea-43ce-8a9c-da8d9e565a20 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:52.108864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-a0bf6644-efde-437e-8bf3-61b708f5e83c None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:52.110103 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a0bf6644-efde-437e-8bf3-61b708f5e83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:52.122548 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1018/2041] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:52.132030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf6644-efde-437e-8bf3-61b708f5e83c None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:52.145343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf6644-efde-437e-8bf3-61b708f5e83c None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:52.151083 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3691b-5a6d-40fe-9a65-2d3b4be74ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:52.151711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3691b-5a6d-40fe-9a65-2d3b4be74ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.151995 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3691b-5a6d-40fe-9a65-2d3b4be74ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.152270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3691b-5a6d-40fe-9a65-2d3b4be74ba5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.152919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3691b-5a6d-40fe-9a65-2d3b4be74ba5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:52.153228 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3691b-5a6d-40fe-9a65-2d3b4be74ba5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:52.155255 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3691b-5a6d-40fe-9a65-2d3b4be74ba5 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:52.168766 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1019/2042] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:52.173288 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3691b-5a6d-40fe-9a65-2d3b4be74ba5 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:52.184733 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3691b-5a6d-40fe-9a65-2d3b4be74ba5 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:52.189576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1d512-a946-44c6-8517-516a4b6c5db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:52.190223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1d512-a946-44c6-8517-516a4b6c5db5 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.190513 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1d512-a946-44c6-8517-516a4b6c5db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.190789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1d512-a946-44c6-8517-516a4b6c5db5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.191460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1d512-a946-44c6-8517-516a4b6c5db5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:52.194011 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1d512-a946-44c6-8517-516a4b6c5db5 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:52.211724 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1024/2043] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:51 2022] POST /identity/v3/users => generated 311 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:52.215944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0a5c5-2eea-43ce-8a9c-da8d9e565a20 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:52.234286 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0a5c5-2eea-43ce-8a9c-da8d9e565a20 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:52.239472 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad37e58-cc16-4d70-b715-d5ee0d23e80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:52.243221 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad37e58-cc16-4d70-b715-d5ee0d23e80e admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.243221 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad37e58-cc16-4d70-b715-d5ee0d23e80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.243221 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad37e58-cc16-4d70-b715-d5ee0d23e80e admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.243221 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad37e58-cc16-4d70-b715-d5ee0d23e80e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:52.243221 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad37e58-cc16-4d70-b715-d5ee0d23e80e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:52.254753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1025/2044] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:52.259370 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad37e58-cc16-4d70-b715-d5ee0d23e80e admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:52.269826 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad37e58-cc16-4d70-b715-d5ee0d23e80e admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:52.276226 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209bda1-97ba-4b4a-b32b-22398c7f5943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:52.277436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209bda1-97ba-4b4a-b32b-22398c7f5943 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.277788 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209bda1-97ba-4b4a-b32b-22398c7f5943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.278018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209bda1-97ba-4b4a-b32b-22398c7f5943 admin admin] PATH_INFO: `/v3/projects/6c6ca5b7ad5749cfb6901c6f043f6f48/users/d79d52ba84564ca7895f2da84e4238db/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.301735 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209bda1-97ba-4b4a-b32b-22398c7f5943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c6ca5b7ad5749cfb6901c6f043f6f48, user_id=d79d52ba84564ca7895f2da84e4238db, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:52.305881 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209bda1-97ba-4b4a-b32b-22398c7f5943 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:52.314210 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1020/2045] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:52 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:26:52.318830 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1d512-a946-44c6-8517-516a4b6c5db5 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:52.385092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1d512-a946-44c6-8517-516a4b6c5db5 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:52.390775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b678bca-b542-4cfd-9699-0c4ac501100f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:52.391431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b678bca-b542-4cfd-9699-0c4ac501100f admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.391723 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b678bca-b542-4cfd-9699-0c4ac501100f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.392000 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b678bca-b542-4cfd-9699-0c4ac501100f admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.392780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b678bca-b542-4cfd-9699-0c4ac501100f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:52.395739 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b678bca-b542-4cfd-9699-0c4ac501100f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:52.401833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1026/2046] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:52 2022] PUT /identity/v3/projects/6c6ca5b7ad5749cfb6901c6f043f6f48/users/d79d52ba84564ca7895f2da84e4238db/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:52.405862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209bda1-97ba-4b4a-b32b-22398c7f5943 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:52.474702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209bda1-97ba-4b4a-b32b-22398c7f5943 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:52.480065 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8f0d7-5968-4fc2-8ba4-c09f711707b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:52.480681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8f0d7-5968-4fc2-8ba4-c09f711707b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.480962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8f0d7-5968-4fc2-8ba4-c09f711707b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.481239 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8f0d7-5968-4fc2-8ba4-c09f711707b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.481921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8f0d7-5968-4fc2-8ba4-c09f711707b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:52.484777 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8f0d7-5968-4fc2-8ba4-c09f711707b9 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:52.487829 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1021/2047] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:52 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:26:52.488868 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b678bca-b542-4cfd-9699-0c4ac501100f admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:52.488997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:52.498609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1027/2048] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:26:52.500165 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b678bca-b542-4cfd-9699-0c4ac501100f admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:52.502794 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8f0d7-5968-4fc2-8ba4-c09f711707b9 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:52.505322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3452e0-32fd-4b36-881e-89faf0417ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:52.506590 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3452e0-32fd-4b36-881e-89faf0417ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.506894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3452e0-32fd-4b36-881e-89faf0417ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.507372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3452e0-32fd-4b36-881e-89faf0417ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.508011 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3452e0-32fd-4b36-881e-89faf0417ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:52.509652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3452e0-32fd-4b36-881e-89faf0417ae9 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:52.513472 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8f0d7-5968-4fc2-8ba4-c09f711707b9 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:52.518658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb7292-2b41-49db-bacb-68c4bf1ffe06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:52.519339 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb7292-2b41-49db-bacb-68c4bf1ffe06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.519619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb7292-2b41-49db-bacb-68c4bf1ffe06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.519891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb7292-2b41-49db-bacb-68c4bf1ffe06 admin admin] PATH_INFO: `/v3/projects/6c6ca5b7ad5749cfb6901c6f043f6f48/users/d79d52ba84564ca7895f2da84e4238db/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.520632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1022/2049] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:52.523429 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb7292-2b41-49db-bacb-68c4bf1ffe06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c6ca5b7ad5749cfb6901c6f043f6f48, user_id=d79d52ba84564ca7895f2da84e4238db, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:52.527575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb7292-2b41-49db-bacb-68c4bf1ffe06 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:52.530287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3452e0-32fd-4b36-881e-89faf0417ae9 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:52.536767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1028/2050] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:52 2022] PUT /identity/v3/projects/6c6ca5b7ad5749cfb6901c6f043f6f48/users/d79d52ba84564ca7895f2da84e4238db/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:52.540914 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3452e0-32fd-4b36-881e-89faf0417ae9 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:52.547005 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f95c3-f300-40d2-b632-7fa25ba47a1f None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.547212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f95c3-f300-40d2-b632-7fa25ba47a1f None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.547506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f95c3-f300-40d2-b632-7fa25ba47a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.587455 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-be5f95c3-f300-40d2-b632-7fa25ba47a1f None None] MFA Rules not processed for user `d79d52ba84564ca7895f2da84e4238db`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:52.589062 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-be5f95c3-f300-40d2-b632-7fa25ba47a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:52.618081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a83eb-c5d7-4587-9fad-05db7207b0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:52.618767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a83eb-c5d7-4587-9fad-05db7207b0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.619168 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a83eb-c5d7-4587-9fad-05db7207b0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.619446 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a83eb-c5d7-4587-9fad-05db7207b0d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.620091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a83eb-c5d7-4587-9fad-05db7207b0d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:52.621792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a83eb-c5d7-4587-9fad-05db7207b0d8 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:52.634545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1023/2051] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:52.639898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a83eb-c5d7-4587-9fad-05db7207b0d8 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:52.654033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a83eb-c5d7-4587-9fad-05db7207b0d8 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:52.665812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c50a43-b824-417f-9b31-c5207fa917d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:52.665812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c50a43-b824-417f-9b31-c5207fa917d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.665812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c50a43-b824-417f-9b31-c5207fa917d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.665812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c50a43-b824-417f-9b31-c5207fa917d9 admin admin] PATH_INFO: `/v3/projects/618ee4f3c61e419f8aa4ccc40a427f51/users/759968dfe3584da6ac7bffa32572c96c/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.670169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1029/2052] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:52 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:26:52.681993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f95c3-f300-40d2-b632-7fa25ba47a1f None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:52.683930 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-be5f95c3-f300-40d2-b632-7fa25ba47a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:52.687051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c50a43-b824-417f-9b31-c5207fa917d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=618ee4f3c61e419f8aa4ccc40a427f51, user_id=759968dfe3584da6ac7bffa32572c96c, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:52.690383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c50a43-b824-417f-9b31-c5207fa917d9 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:52.744661 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f95c3-f300-40d2-b632-7fa25ba47a1f None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:52.748775 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f2c6f-7553-444d-bf94-aece6e57d019 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:52.749384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f2c6f-7553-444d-bf94-aece6e57d019 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.749667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f2c6f-7553-444d-bf94-aece6e57d019 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.749945 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f2c6f-7553-444d-bf94-aece6e57d019 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.759173 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f2c6f-7553-444d-bf94-aece6e57d019 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:52.760607 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f2c6f-7553-444d-bf94-aece6e57d019 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:52.770723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1030/2053] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:52 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:52.788612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1024/2054] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:52 2022] PUT /identity/v3/projects/618ee4f3c61e419f8aa4ccc40a427f51/users/759968dfe3584da6ac7bffa32572c96c/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:52.791134 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6896c02-5130-4b19-a734-8bb764da5656 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.791572 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6896c02-5130-4b19-a734-8bb764da5656 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.791991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6896c02-5130-4b19-a734-8bb764da5656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.823271 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-f6896c02-5130-4b19-a734-8bb764da5656 None None] MFA Rules not processed for user `759968dfe3584da6ac7bffa32572c96c`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:52.825111 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f6896c02-5130-4b19-a734-8bb764da5656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:52.922108 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1031/2055] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:52.926546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c50a43-b824-417f-9b31-c5207fa917d9 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:52.932586 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:52.992327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c50a43-b824-417f-9b31-c5207fa917d9 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:52.997274 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df551dad-eb39-4b3f-8b2f-218c5c40aac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:52.997917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df551dad-eb39-4b3f-8b2f-218c5c40aac1 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.998201 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df551dad-eb39-4b3f-8b2f-218c5c40aac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.998481 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df551dad-eb39-4b3f-8b2f-218c5c40aac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.999188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df551dad-eb39-4b3f-8b2f-218c5c40aac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:53.001158 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df551dad-eb39-4b3f-8b2f-218c5c40aac1 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:53.103866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1025/2056] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:52 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:53.107627 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6896c02-5130-4b19-a734-8bb764da5656 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:53.114344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:53.120997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6896c02-5130-4b19-a734-8bb764da5656 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:53.186188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb5ef3c-143f-4d6f-80e7-c6d043e22b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:53.186930 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5ef3c-143f-4d6f-80e7-c6d043e22b80 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:53.187223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5ef3c-143f-4d6f-80e7-c6d043e22b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:53.187500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5ef3c-143f-4d6f-80e7-c6d043e22b80 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:53.188146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb5ef3c-143f-4d6f-80e7-c6d043e22b80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:53.190622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb5ef3c-143f-4d6f-80e7-c6d043e22b80 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:53.275691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df551dad-eb39-4b3f-8b2f-218c5c40aac1 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:53.298581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1032/2057] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:53 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:53.299775 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb5ef3c-143f-4d6f-80e7-c6d043e22b80 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:53.310647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb5ef3c-143f-4d6f-80e7-c6d043e22b80 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:53.314994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede4669-c36c-4143-aae0-6ae66a88f3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:53.315591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede4669-c36c-4143-aae0-6ae66a88f3d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:53.315871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede4669-c36c-4143-aae0-6ae66a88f3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:53.316140 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede4669-c36c-4143-aae0-6ae66a88f3d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:53.316943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede4669-c36c-4143-aae0-6ae66a88f3d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:53.319525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede4669-c36c-4143-aae0-6ae66a88f3d5 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:53.338724 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df551dad-eb39-4b3f-8b2f-218c5c40aac1 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:53.343317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192faf8-05ac-468f-a67a-f289a49c9d71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:53.344590 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192faf8-05ac-468f-a67a-f289a49c9d71 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:53.345461 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192faf8-05ac-468f-a67a-f289a49c9d71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:53.346018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192faf8-05ac-468f-a67a-f289a49c9d71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:53.348872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1033/2058] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:26:53.349119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e192faf8-05ac-468f-a67a-f289a49c9d71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:53.358996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede4669-c36c-4143-aae0-6ae66a88f3d5 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:53.371877 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede4669-c36c-4143-aae0-6ae66a88f3d5 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:53.379548 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24288ff8-1b55-496e-8e86-089a7fed512f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:53.380591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24288ff8-1b55-496e-8e86-089a7fed512f admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:53.381905 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24288ff8-1b55-496e-8e86-089a7fed512f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:53.383125 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24288ff8-1b55-496e-8e86-089a7fed512f admin admin] PATH_INFO: `/v3/projects/7c06411eeab84356bc92eab01874bd9c/users/8b0a4338125741caa2f77ead42d2eb2e/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:53.408481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24288ff8-1b55-496e-8e86-089a7fed512f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c06411eeab84356bc92eab01874bd9c, user_id=8b0a4338125741caa2f77ead42d2eb2e, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:53.415039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24288ff8-1b55-496e-8e86-089a7fed512f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:53.436771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192faf8-05ac-468f-a67a-f289a49c9d71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:53.438480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192faf8-05ac-468f-a67a-f289a49c9d71 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:53.450903 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1026/2059] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:53 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:26:53.516231 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1034/2060] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:53 2022] PUT /identity/v3/projects/7c06411eeab84356bc92eab01874bd9c/users/8b0a4338125741caa2f77ead42d2eb2e/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:53.520457 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192faf8-05ac-468f-a67a-f289a49c9d71 service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:53.544912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192faf8-05ac-468f-a67a-f289a49c9d71 service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:53.629407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:53.644680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61ee06-f3ff-46cd-a317-978c4cc3ebaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:53.645301 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61ee06-f3ff-46cd-a317-978c4cc3ebaa admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:53.645641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61ee06-f3ff-46cd-a317-978c4cc3ebaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:53.645978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61ee06-f3ff-46cd-a317-978c4cc3ebaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:53.647414 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61ee06-f3ff-46cd-a317-978c4cc3ebaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:53.649078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61ee06-f3ff-46cd-a317-978c4cc3ebaa admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:53.663445 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1027/2061] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:53.665106 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24288ff8-1b55-496e-8e86-089a7fed512f admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:53.678737 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24288ff8-1b55-496e-8e86-089a7fed512f admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:53.683825 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8afea-06f9-45f2-8cd9-3c86023539b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:53.684582 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8afea-06f9-45f2-8cd9-3c86023539b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:53.684862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8afea-06f9-45f2-8cd9-3c86023539b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:53.685208 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8afea-06f9-45f2-8cd9-3c86023539b5 admin admin] PATH_INFO: `/v3/projects/7c06411eeab84356bc92eab01874bd9c/users/8b0a4338125741caa2f77ead42d2eb2e/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:53.687490 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8afea-06f9-45f2-8cd9-3c86023539b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c06411eeab84356bc92eab01874bd9c, user_id=8b0a4338125741caa2f77ead42d2eb2e, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:53.690517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8afea-06f9-45f2-8cd9-3c86023539b5 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:53.703579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1035/2062] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:53 2022] PUT /identity/v3/projects/7c06411eeab84356bc92eab01874bd9c/users/8b0a4338125741caa2f77ead42d2eb2e/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:53.714300 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffabfa-308e-43c6-b9e1-a9f6f7773855 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:53.715163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffabfa-308e-43c6-b9e1-a9f6f7773855 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:53.715542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffabfa-308e-43c6-b9e1-a9f6f7773855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:53.750764 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-78ffabfa-308e-43c6-b9e1-a9f6f7773855 None None] MFA Rules not processed for user `8b0a4338125741caa2f77ead42d2eb2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:53.752193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-78ffabfa-308e-43c6-b9e1-a9f6f7773855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:53.841446 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1028/2063] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:26:53.854814 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8afea-06f9-45f2-8cd9-3c86023539b5 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:53.856660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-91d8afea-06f9-45f2-8cd9-3c86023539b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:53.922913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8afea-06f9-45f2-8cd9-3c86023539b5 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:53.926629 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03e71c-e58d-4bfc-83ae-1897df949a09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:53.927306 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03e71c-e58d-4bfc-83ae-1897df949a09 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:53.927591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03e71c-e58d-4bfc-83ae-1897df949a09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:53.930223 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03e71c-e58d-4bfc-83ae-1897df949a09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:53.946092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03e71c-e58d-4bfc-83ae-1897df949a09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:53.947609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03e71c-e58d-4bfc-83ae-1897df949a09 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:53.958319 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1036/2064] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:26:54.156149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffabfa-308e-43c6-b9e1-a9f6f7773855 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:54.167759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffabfa-308e-43c6-b9e1-a9f6f7773855 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:54.177021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c323a7-6115-46c2-8740-c17e97bd0e9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:54.177021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c323a7-6115-46c2-8740-c17e97bd0e9e service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:54.177021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c323a7-6115-46c2-8740-c17e97bd0e9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:54.177021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c323a7-6115-46c2-8740-c17e97bd0e9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:54.253081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c323a7-6115-46c2-8740-c17e97bd0e9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:54.254858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c323a7-6115-46c2-8740-c17e97bd0e9e service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:54.273786 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1029/2065] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:54.871645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03e71c-e58d-4bfc-83ae-1897df949a09 service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:54.926369 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:54.941125 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03e71c-e58d-4bfc-83ae-1897df949a09 service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:54.946459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5388a-26f0-40e5-9bd2-03b64bf58bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:54.947137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5388a-26f0-40e5-9bd2-03b64bf58bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:54.947425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5388a-26f0-40e5-9bd2-03b64bf58bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:54.947698 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5388a-26f0-40e5-9bd2-03b64bf58bd0 admin admin] PATH_INFO: `/v3/users/759968dfe3584da6ac7bffa32572c96c` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:54.948867 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5388a-26f0-40e5-9bd2-03b64bf58bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=759968dfe3584da6ac7bffa32572c96c)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:54.950782 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5388a-26f0-40e5-9bd2-03b64bf58bd0 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:55.183320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-10c5388a-26f0-40e5-9bd2-03b64bf58bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '759968dfe3584da6ac7bffa32572c96c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:55.197265 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-10c5388a-26f0-40e5-9bd2-03b64bf58bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '759968dfe3584da6ac7bffa32572c96c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:55.260675 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-10c5388a-26f0-40e5-9bd2-03b64bf58bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '759968dfe3584da6ac7bffa32572c96c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:55.275467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-10c5388a-26f0-40e5-9bd2-03b64bf58bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '759968dfe3584da6ac7bffa32572c96c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:55.276455 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-10c5388a-26f0-40e5-9bd2-03b64bf58bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '759968dfe3584da6ac7bffa32572c96c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:55.289564 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1037/2066] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:54 2022] DELETE /identity/v3/users/759968dfe3584da6ac7bffa32572c96c => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:55.294181 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41b522-c63d-4a22-ac6d-d5a538bf7cf3 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:55.294694 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41b522-c63d-4a22-ac6d-d5a538bf7cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:55.294849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41b522-c63d-4a22-ac6d-d5a538bf7cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:55.320458 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-db41b522-c63d-4a22-ac6d-d5a538bf7cf3 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:55.321735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-db41b522-c63d-4a22-ac6d-d5a538bf7cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:55.425437 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1030/2067] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:55.435103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5388a-26f0-40e5-9bd2-03b64bf58bd0 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:55.437463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-10c5388a-26f0-40e5-9bd2-03b64bf58bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:55.525051 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5388a-26f0-40e5-9bd2-03b64bf58bd0 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:55.530976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497bd695-023f-4886-8919-555877640cb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:55.530976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bd695-023f-4886-8919-555877640cb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:55.531250 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bd695-023f-4886-8919-555877640cb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:55.531596 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bd695-023f-4886-8919-555877640cb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:55.544999 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bd695-023f-4886-8919-555877640cb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:55.546532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bd695-023f-4886-8919-555877640cb4 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:55.559785 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1038/2068] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:56.221011 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41b522-c63d-4a22-ac6d-d5a538bf7cf3 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:56.222796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-db41b522-c63d-4a22-ac6d-d5a538bf7cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:56.245006 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:56.258596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41b522-c63d-4a22-ac6d-d5a538bf7cf3 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:56.264221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db74cd-3b6c-4752-a96a-a6690441f31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:56.265028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db74cd-3b6c-4752-a96a-a6690441f31a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:56.265433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db74cd-3b6c-4752-a96a-a6690441f31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:56.265851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db74cd-3b6c-4752-a96a-a6690441f31a admin admin] PATH_INFO: `/v3/projects/618ee4f3c61e419f8aa4ccc40a427f51` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:56.267506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db74cd-3b6c-4752-a96a-a6690441f31a admin admin] RBAC: Authorizing `identity:delete_project(project_id=618ee4f3c61e419f8aa4ccc40a427f51)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:56.273839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db74cd-3b6c-4752-a96a-a6690441f31a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:56.414677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-73db74cd-3b6c-4752-a96a-a6690441f31a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '618ee4f3c61e419f8aa4ccc40a427f51', 'request_id': 'req-73db74cd-3b6c-4752-a96a-a6690441f31a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:56.421817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-73db74cd-3b6c-4752-a96a-a6690441f31a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '618ee4f3c61e419f8aa4ccc40a427f51', 'request_id': 'req-73db74cd-3b6c-4752-a96a-a6690441f31a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:56.430365 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-73db74cd-3b6c-4752-a96a-a6690441f31a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '618ee4f3c61e419f8aa4ccc40a427f51', 'request_id': 'req-73db74cd-3b6c-4752-a96a-a6690441f31a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:56.553960 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-73db74cd-3b6c-4752-a96a-a6690441f31a admin admin] The token cache is being invalidate because project 618ee4f3c61e419f8aa4ccc40a427f51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:56.554354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-73db74cd-3b6c-4752-a96a-a6690441f31a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:56.555172 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-73db74cd-3b6c-4752-a96a-a6690441f31a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:56.557417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1031/2069] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:56 2022] DELETE /identity/v3/projects/618ee4f3c61e419f8aa4ccc40a427f51 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:56.561943 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497bd695-023f-4886-8919-555877640cb4 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:56.575883 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497bd695-023f-4886-8919-555877640cb4 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:56.670499 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ba09b-d31a-4ab4-b135-928aad4d1f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:56.672695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ba09b-d31a-4ab4-b135-928aad4d1f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:56.673063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ba09b-d31a-4ab4-b135-928aad4d1f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:56.673414 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ba09b-d31a-4ab4-b135-928aad4d1f8e admin admin] PATH_INFO: `/v3/users/8b0a4338125741caa2f77ead42d2eb2e` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:56.674676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ba09b-d31a-4ab4-b135-928aad4d1f8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b0a4338125741caa2f77ead42d2eb2e)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:56.676845 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ba09b-d31a-4ab4-b135-928aad4d1f8e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:56.690507 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:56.914520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7a3ba09b-d31a-4ab4-b135-928aad4d1f8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b0a4338125741caa2f77ead42d2eb2e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:56.929655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7a3ba09b-d31a-4ab4-b135-928aad4d1f8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b0a4338125741caa2f77ead42d2eb2e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:57.027022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7a3ba09b-d31a-4ab4-b135-928aad4d1f8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b0a4338125741caa2f77ead42d2eb2e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:57.043831 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7a3ba09b-d31a-4ab4-b135-928aad4d1f8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b0a4338125741caa2f77ead42d2eb2e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:57.045911 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7a3ba09b-d31a-4ab4-b135-928aad4d1f8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b0a4338125741caa2f77ead42d2eb2e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:57.067123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1039/2070] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:56 2022] DELETE /identity/v3/users/8b0a4338125741caa2f77ead42d2eb2e => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:57.733367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db74cd-3b6c-4752-a96a-a6690441f31a admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:57.734953 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-73db74cd-3b6c-4752-a96a-a6690441f31a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:57.756828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db74cd-3b6c-4752-a96a-a6690441f31a admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:57.851700 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456e409-3db0-4f72-8c9a-ba8130aa84a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:57.852640 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456e409-3db0-4f72-8c9a-ba8130aa84a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:57.853340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456e409-3db0-4f72-8c9a-ba8130aa84a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:57.854496 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456e409-3db0-4f72-8c9a-ba8130aa84a4 admin admin] PATH_INFO: `/v3/projects/7c06411eeab84356bc92eab01874bd9c` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:57.856264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456e409-3db0-4f72-8c9a-ba8130aa84a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c06411eeab84356bc92eab01874bd9c)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:57.858486 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456e409-3db0-4f72-8c9a-ba8130aa84a4 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:57.982227 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a456e409-3db0-4f72-8c9a-ba8130aa84a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c06411eeab84356bc92eab01874bd9c', 'request_id': 'req-a456e409-3db0-4f72-8c9a-ba8130aa84a4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:57.993951 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a456e409-3db0-4f72-8c9a-ba8130aa84a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c06411eeab84356bc92eab01874bd9c', 'request_id': 'req-a456e409-3db0-4f72-8c9a-ba8130aa84a4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:58.004715 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a456e409-3db0-4f72-8c9a-ba8130aa84a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c06411eeab84356bc92eab01874bd9c', 'request_id': 'req-a456e409-3db0-4f72-8c9a-ba8130aa84a4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:58.132846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a456e409-3db0-4f72-8c9a-ba8130aa84a4 admin admin] The token cache is being invalidate because project 7c06411eeab84356bc92eab01874bd9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:58.133493 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a456e409-3db0-4f72-8c9a-ba8130aa84a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:58.134707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a456e409-3db0-4f72-8c9a-ba8130aa84a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:58.136695 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1032/2071] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:57 2022] DELETE /identity/v3/projects/7c06411eeab84356bc92eab01874bd9c => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:58.142373 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fdff3-ccf2-4167-a89e-bcab5181ad64 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.143355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fdff3-ccf2-4167-a89e-bcab5181ad64 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.143836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fdff3-ccf2-4167-a89e-bcab5181ad64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.169669 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-978fdff3-ccf2-4167-a89e-bcab5181ad64 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:58.170999 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-978fdff3-ccf2-4167-a89e-bcab5181ad64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:58.243044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1040/2072] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:58.252260 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95ff5a-1ce7-49ba-97f0-e3e54fd0de3e None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.252546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95ff5a-1ce7-49ba-97f0-e3e54fd0de3e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.253621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95ff5a-1ce7-49ba-97f0-e3e54fd0de3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.287574 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-7d95ff5a-1ce7-49ba-97f0-e3e54fd0de3e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:58.289589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-7d95ff5a-1ce7-49ba-97f0-e3e54fd0de3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:58.301135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7e821-343a-4443-8dec-1d2e4d5348ae None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.301437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7e821-343a-4443-8dec-1d2e4d5348ae None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.301686 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7e821-343a-4443-8dec-1d2e4d5348ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.309075 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1033/2073] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:58.319010 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95ff5a-1ce7-49ba-97f0-e3e54fd0de3e None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.339570 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95ff5a-1ce7-49ba-97f0-e3e54fd0de3e None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.358413 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3c596-74c5-41cb-8bea-d5f24345a310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.360596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3c596-74c5-41cb-8bea-d5f24345a310 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.360596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3c596-74c5-41cb-8bea-d5f24345a310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.360596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3c596-74c5-41cb-8bea-d5f24345a310 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.360596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3c596-74c5-41cb-8bea-d5f24345a310 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:58.360935 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3c596-74c5-41cb-8bea-d5f24345a310 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.363652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3c596-74c5-41cb-8bea-d5f24345a310 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:58.367932 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-8ec7e821-343a-4443-8dec-1d2e4d5348ae None None] MFA Rules not processed for user `be55f0bb21f044b0a8346cf953b1650e`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:58.368430 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8ec7e821-343a-4443-8dec-1d2e4d5348ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:58.376954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:58.390820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1034/2074] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:58.391932 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3c596-74c5-41cb-8bea-d5f24345a310 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.403105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3c596-74c5-41cb-8bea-d5f24345a310 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.408913 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4b1ed-c0f2-4e3d-b5b6-186083c778aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.409605 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4b1ed-c0f2-4e3d-b5b6-186083c778aa admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.409958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4b1ed-c0f2-4e3d-b5b6-186083c778aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.410300 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4b1ed-c0f2-4e3d-b5b6-186083c778aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.411035 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4b1ed-c0f2-4e3d-b5b6-186083c778aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.413282 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4b1ed-c0f2-4e3d-b5b6-186083c778aa admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:58.466539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1041/2075] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:58.471447 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7e821-343a-4443-8dec-1d2e4d5348ae None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.472802 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8ec7e821-343a-4443-8dec-1d2e4d5348ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:58.489910 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7e821-343a-4443-8dec-1d2e4d5348ae None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.495940 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81291a06-f11c-4ccc-b6ef-bce7663cf16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.496682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81291a06-f11c-4ccc-b6ef-bce7663cf16f admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.496990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81291a06-f11c-4ccc-b6ef-bce7663cf16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.497273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81291a06-f11c-4ccc-b6ef-bce7663cf16f admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.497927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81291a06-f11c-4ccc-b6ef-bce7663cf16f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.499887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81291a06-f11c-4ccc-b6ef-bce7663cf16f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:58.520464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:58.536213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1035/2076] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:58 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:58.540170 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4b1ed-c0f2-4e3d-b5b6-186083c778aa admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.611090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1042/2077] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:58 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:26:58.612013 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81291a06-f11c-4ccc-b6ef-bce7663cf16f admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.615965 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4b1ed-c0f2-4e3d-b5b6-186083c778aa admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.686043 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf059f7-e8b8-40b7-a5a7-2caa6231c77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.687102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf059f7-e8b8-40b7-a5a7-2caa6231c77a admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.691667 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf059f7-e8b8-40b7-a5a7-2caa6231c77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.691667 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf059f7-e8b8-40b7-a5a7-2caa6231c77a admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.691667 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf059f7-e8b8-40b7-a5a7-2caa6231c77a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.691667 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf059f7-e8b8-40b7-a5a7-2caa6231c77a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:58.709564 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81291a06-f11c-4ccc-b6ef-bce7663cf16f admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.718486 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674a81e-6289-470b-afd9-cf2077511ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.719178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674a81e-6289-470b-afd9-cf2077511ccf admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.719466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674a81e-6289-470b-afd9-cf2077511ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.719742 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674a81e-6289-470b-afd9-cf2077511ccf admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.720390 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674a81e-6289-470b-afd9-cf2077511ccf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.722404 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674a81e-6289-470b-afd9-cf2077511ccf admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:58.899437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1043/2078] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:58 2022] POST /identity/v3/users => generated 294 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:58.904901 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674a81e-6289-470b-afd9-cf2077511ccf admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.936879 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674a81e-6289-470b-afd9-cf2077511ccf admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.939891 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1036/2079] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:58 2022] POST /identity/v3/users => generated 292 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:26:58.942341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf059f7-e8b8-40b7-a5a7-2caa6231c77a admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.955240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6617f-264b-4aee-af51-ba91bc6e84b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.955877 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6617f-264b-4aee-af51-ba91bc6e84b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.956187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6617f-264b-4aee-af51-ba91bc6e84b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.956475 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6617f-264b-4aee-af51-ba91bc6e84b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.957122 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6617f-264b-4aee-af51-ba91bc6e84b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.958955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6617f-264b-4aee-af51-ba91bc6e84b0 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:58.963172 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf059f7-e8b8-40b7-a5a7-2caa6231c77a admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.978622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1044/2080] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:26:58.980985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f99bb4-8802-43aa-91f2-6ed037b6bbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.982153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f99bb4-8802-43aa-91f2-6ed037b6bbbe admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.982289 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6617f-264b-4aee-af51-ba91bc6e84b0 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.982706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f99bb4-8802-43aa-91f2-6ed037b6bbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.983157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f99bb4-8802-43aa-91f2-6ed037b6bbbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.983972 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f99bb4-8802-43aa-91f2-6ed037b6bbbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.985847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f99bb4-8802-43aa-91f2-6ed037b6bbbe admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:59.008586 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1037/2081] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:59.009432 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6617f-264b-4aee-af51-ba91bc6e84b0 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:59.010415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f99bb4-8802-43aa-91f2-6ed037b6bbbe admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:59.013728 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96065860-f097-49f1-b6c7-e2da21826b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:59.015459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96065860-f097-49f1-b6c7-e2da21826b57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:59.015459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96065860-f097-49f1-b6c7-e2da21826b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:59.015759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96065860-f097-49f1-b6c7-e2da21826b57 admin admin] PATH_INFO: `/v3/projects/f1d33e3eb5ee498f9422a177338b0f9d/users/d7ac3fe714fb48459a3524d4a458de62/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:59.026310 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f99bb4-8802-43aa-91f2-6ed037b6bbbe admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:59.031469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28b016-d8f2-47ef-a046-ab7dc8d9e58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:59.032435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28b016-d8f2-47ef-a046-ab7dc8d9e58f admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:59.032872 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28b016-d8f2-47ef-a046-ab7dc8d9e58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:59.033149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28b016-d8f2-47ef-a046-ab7dc8d9e58f admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:59.033791 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28b016-d8f2-47ef-a046-ab7dc8d9e58f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:59.035525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28b016-d8f2-47ef-a046-ab7dc8d9e58f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:59.036160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96065860-f097-49f1-b6c7-e2da21826b57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1d33e3eb5ee498f9422a177338b0f9d, user_id=d7ac3fe714fb48459a3524d4a458de62, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:59.039604 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96065860-f097-49f1-b6c7-e2da21826b57 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:59.049416 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1038/2082] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:59.054196 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28b016-d8f2-47ef-a046-ab7dc8d9e58f admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:59.064633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28b016-d8f2-47ef-a046-ab7dc8d9e58f admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:59.070851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b1399-0b65-4c1a-8eba-4078f4d4dc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:59.071744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b1399-0b65-4c1a-8eba-4078f4d4dc10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:59.072163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b1399-0b65-4c1a-8eba-4078f4d4dc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:59.072643 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b1399-0b65-4c1a-8eba-4078f4d4dc10 admin admin] PATH_INFO: `/v3/projects/3f96cd76884c4f72a7647ca56fd1c761/users/b0185e15d06f4a7c981dbdac309fcddd/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:59.091755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b1399-0b65-4c1a-8eba-4078f4d4dc10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f96cd76884c4f72a7647ca56fd1c761, user_id=b0185e15d06f4a7c981dbdac309fcddd, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:59.094414 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b1399-0b65-4c1a-8eba-4078f4d4dc10 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:59.134204 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1045/2083] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:58 2022] PUT /identity/v3/projects/f1d33e3eb5ee498f9422a177338b0f9d/users/d7ac3fe714fb48459a3524d4a458de62/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:59.138249 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96065860-f097-49f1-b6c7-e2da21826b57 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:59.174961 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1039/2084] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:59 2022] PUT /identity/v3/projects/3f96cd76884c4f72a7647ca56fd1c761/users/b0185e15d06f4a7c981dbdac309fcddd/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:59.190605 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db1a50-b836-40c5-ba56-ed661f49ff24 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:59.191263 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db1a50-b836-40c5-ba56-ed661f49ff24 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:59.191544 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db1a50-b836-40c5-ba56-ed661f49ff24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:59.207936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96065860-f097-49f1-b6c7-e2da21826b57 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:59.222628 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-b4db1a50-b836-40c5-ba56-ed661f49ff24 None None] MFA Rules not processed for user `b0185e15d06f4a7c981dbdac309fcddd`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:59.224125 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b4db1a50-b836-40c5-ba56-ed661f49ff24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:59.237792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:59.264257 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:59.270569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2870d-8084-4016-9fc4-069e07c7b8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:59.271561 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2870d-8084-4016-9fc4-069e07c7b8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:59.271889 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2870d-8084-4016-9fc4-069e07c7b8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:59.272253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2870d-8084-4016-9fc4-069e07c7b8a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:59.273282 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2870d-8084-4016-9fc4-069e07c7b8a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:59.275388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2870d-8084-4016-9fc4-069e07c7b8a4 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:59.289862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1046/2085] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:26:59.290747 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2870d-8084-4016-9fc4-069e07c7b8a4 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:59.293976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1040/2086] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:26:59.298120 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db1a50-b836-40c5-ba56-ed661f49ff24 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:59.307585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2870d-8084-4016-9fc4-069e07c7b8a4 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:59.308796 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97f12e-23c6-4e35-affd-ff2720636dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:59.309475 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97f12e-23c6-4e35-affd-ff2720636dbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:59.309761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97f12e-23c6-4e35-affd-ff2720636dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:59.310041 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97f12e-23c6-4e35-affd-ff2720636dbb admin admin] PATH_INFO: `/v3/projects/f1d33e3eb5ee498f9422a177338b0f9d/users/d7ac3fe714fb48459a3524d4a458de62/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:59.310678 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db1a50-b836-40c5-ba56-ed661f49ff24 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:59.312077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97f12e-23c6-4e35-affd-ff2720636dbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1d33e3eb5ee498f9422a177338b0f9d, user_id=d7ac3fe714fb48459a3524d4a458de62, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:59.314759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97f12e-23c6-4e35-affd-ff2720636dbb admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:59.315454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e33227-68f6-4486-ae68-40bb7a70320c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:59.316108 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e33227-68f6-4486-ae68-40bb7a70320c admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:59.316394 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e33227-68f6-4486-ae68-40bb7a70320c admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:59.316690 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e33227-68f6-4486-ae68-40bb7a70320c admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:59.317342 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e33227-68f6-4486-ae68-40bb7a70320c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:59.320109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e33227-68f6-4486-ae68-40bb7a70320c admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:59.327077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1047/2087] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:59 2022] PUT /identity/v3/projects/f1d33e3eb5ee498f9422a177338b0f9d/users/d7ac3fe714fb48459a3524d4a458de62/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:59.444655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1041/2088] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:59 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:26:59.448579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97f12e-23c6-4e35-affd-ff2720636dbb admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:59.532281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97f12e-23c6-4e35-affd-ff2720636dbb admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:59.602520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a91a5-b6e8-4452-b478-1ae672884409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:59.603110 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a91a5-b6e8-4452-b478-1ae672884409 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:59.603396 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a91a5-b6e8-4452-b478-1ae672884409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:59.603668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a91a5-b6e8-4452-b478-1ae672884409 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:59.604319 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a91a5-b6e8-4452-b478-1ae672884409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:59.606332 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a91a5-b6e8-4452-b478-1ae672884409 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:59.690524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1048/2089] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:59 2022] POST /identity/v3/users => generated 292 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:59.694449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e33227-68f6-4486-ae68-40bb7a70320c admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:59.707437 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e33227-68f6-4486-ae68-40bb7a70320c admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:59.717207 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caca170-9776-48f7-8299-8f739d635671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:59.718083 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caca170-9776-48f7-8299-8f739d635671 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:59.718535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caca170-9776-48f7-8299-8f739d635671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:59.719064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caca170-9776-48f7-8299-8f739d635671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:59.725216 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caca170-9776-48f7-8299-8f739d635671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:59.725216 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caca170-9776-48f7-8299-8f739d635671 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:59.735485 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1042/2090] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:59.739799 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a91a5-b6e8-4452-b478-1ae672884409 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:59.753954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a91a5-b6e8-4452-b478-1ae672884409 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:59.758162 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1a860-c8e2-4a60-a5d7-145f450b8d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:59.758836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1a860-c8e2-4a60-a5d7-145f450b8d15 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:59.759225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1a860-c8e2-4a60-a5d7-145f450b8d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:59.759504 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1a860-c8e2-4a60-a5d7-145f450b8d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:59.760484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1a860-c8e2-4a60-a5d7-145f450b8d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:59.762169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1a860-c8e2-4a60-a5d7-145f450b8d15 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:59.774312 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1049/2091] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:26:59.778913 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caca170-9776-48f7-8299-8f739d635671 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:59.789629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caca170-9776-48f7-8299-8f739d635671 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:59.794190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644632e7-0e2e-41d5-8572-daac6cab3b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:59.794846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644632e7-0e2e-41d5-8572-daac6cab3b07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:59.795914 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644632e7-0e2e-41d5-8572-daac6cab3b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:59.796205 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644632e7-0e2e-41d5-8572-daac6cab3b07 admin admin] PATH_INFO: `/v3/projects/4b919359e41f4c8c8e442376912e228a/users/6e8f8fcda8754323ba4217a6fcf61ef6/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:59.812555 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644632e7-0e2e-41d5-8572-daac6cab3b07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b919359e41f4c8c8e442376912e228a, user_id=6e8f8fcda8754323ba4217a6fcf61ef6, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:59.815413 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644632e7-0e2e-41d5-8572-daac6cab3b07 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:59.932519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1043/2092] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:59 2022] PUT /identity/v3/projects/4b919359e41f4c8c8e442376912e228a/users/6e8f8fcda8754323ba4217a6fcf61ef6/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:59.933867 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b686d-05ec-4059-bec9-67d797652db3 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:59.934716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b686d-05ec-4059-bec9-67d797652db3 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:59.934716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b686d-05ec-4059-bec9-67d797652db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:59.966161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-256b686d-05ec-4059-bec9-67d797652db3 None None] MFA Rules not processed for user `6e8f8fcda8754323ba4217a6fcf61ef6`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:59.967317 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-256b686d-05ec-4059-bec9-67d797652db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:00.038732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:00.050690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1050/2093] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:26:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:00.056942 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644632e7-0e2e-41d5-8572-daac6cab3b07 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:00.123657 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644632e7-0e2e-41d5-8572-daac6cab3b07 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:00.129239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd060a-f9d4-46dd-b28b-23e56c0ef6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:00.130173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd060a-f9d4-46dd-b28b-23e56c0ef6e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:00.130173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd060a-f9d4-46dd-b28b-23e56c0ef6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:00.130439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd060a-f9d4-46dd-b28b-23e56c0ef6e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:00.131569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd060a-f9d4-46dd-b28b-23e56c0ef6e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:00.133664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd060a-f9d4-46dd-b28b-23e56c0ef6e7 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:00.265967 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1044/2094] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:00 2022] POST /identity/v3/projects => generated 385 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:00.269861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b686d-05ec-4059-bec9-67d797652db3 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:00.294958 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b686d-05ec-4059-bec9-67d797652db3 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:00.361773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77a067-f18d-4a28-ae6c-f32878594fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:00.362488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77a067-f18d-4a28-ae6c-f32878594fac admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:00.362845 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77a067-f18d-4a28-ae6c-f32878594fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:00.363150 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77a067-f18d-4a28-ae6c-f32878594fac admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:00.363972 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77a067-f18d-4a28-ae6c-f32878594fac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:00.366403 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77a067-f18d-4a28-ae6c-f32878594fac admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:00.536086 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1051/2095] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:00 2022] POST /identity/v3/users => generated 299 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:00.540289 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd060a-f9d4-46dd-b28b-23e56c0ef6e7 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:00.551090 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd060a-f9d4-46dd-b28b-23e56c0ef6e7 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:00.556059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7b9b5-1014-4729-b058-bec2b3512018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:00.556685 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7b9b5-1014-4729-b058-bec2b3512018 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:00.556988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7b9b5-1014-4729-b058-bec2b3512018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:00.557242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7b9b5-1014-4729-b058-bec2b3512018 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:00.557952 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7b9b5-1014-4729-b058-bec2b3512018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:00.559606 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7b9b5-1014-4729-b058-bec2b3512018 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:00.575019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1045/2096] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:00.576315 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77a067-f18d-4a28-ae6c-f32878594fac admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:00.589136 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77a067-f18d-4a28-ae6c-f32878594fac admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:00.595109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1383da5-5cb1-4eed-a55b-c56bccc80425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:00.595513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1383da5-5cb1-4eed-a55b-c56bccc80425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:00.595898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1383da5-5cb1-4eed-a55b-c56bccc80425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:00.596273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1383da5-5cb1-4eed-a55b-c56bccc80425 admin admin] PATH_INFO: `/v3/projects/c01d926e8e2d4b06a21471e3413ed478/users/b2d2a866714e47f3ab7e4d47bdae1a9b/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:00.616281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1383da5-5cb1-4eed-a55b-c56bccc80425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c01d926e8e2d4b06a21471e3413ed478, user_id=b2d2a866714e47f3ab7e4d47bdae1a9b, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:00.620154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1383da5-5cb1-4eed-a55b-c56bccc80425 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:00.710219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1052/2097] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:00 2022] PUT /identity/v3/projects/c01d926e8e2d4b06a21471e3413ed478/users/b2d2a866714e47f3ab7e4d47bdae1a9b/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:00.714056 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7b9b5-1014-4729-b058-bec2b3512018 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:00.724426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7b9b5-1014-4729-b058-bec2b3512018 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:00.761136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:00.792213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf2ab2-df30-4138-8c5f-3dba71026251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:00.792774 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf2ab2-df30-4138-8c5f-3dba71026251 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:00.793059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf2ab2-df30-4138-8c5f-3dba71026251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:00.793337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf2ab2-df30-4138-8c5f-3dba71026251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:00.794202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf2ab2-df30-4138-8c5f-3dba71026251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:00.795945 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf2ab2-df30-4138-8c5f-3dba71026251 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:00.811364 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1046/2098] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:00.812811 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1383da5-5cb1-4eed-a55b-c56bccc80425 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:00.834626 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1383da5-5cb1-4eed-a55b-c56bccc80425 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:00.841942 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbd6fc-2d2c-42f6-8127-e2c33d72bb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:00.843092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbd6fc-2d2c-42f6-8127-e2c33d72bb64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:00.843642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbd6fc-2d2c-42f6-8127-e2c33d72bb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:00.844114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbd6fc-2d2c-42f6-8127-e2c33d72bb64 admin admin] PATH_INFO: `/v3/projects/c01d926e8e2d4b06a21471e3413ed478/users/b2d2a866714e47f3ab7e4d47bdae1a9b/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:00.846247 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbd6fc-2d2c-42f6-8127-e2c33d72bb64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c01d926e8e2d4b06a21471e3413ed478, user_id=b2d2a866714e47f3ab7e4d47bdae1a9b, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:00.848988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbd6fc-2d2c-42f6-8127-e2c33d72bb64 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:00.867566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1053/2099] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:00 2022] PUT /identity/v3/projects/c01d926e8e2d4b06a21471e3413ed478/users/b2d2a866714e47f3ab7e4d47bdae1a9b/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:00.873292 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0131aa3-1d42-4896-a1c0-5b3580981c61 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:00.873580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0131aa3-1d42-4896-a1c0-5b3580981c61 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:00.873857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0131aa3-1d42-4896-a1c0-5b3580981c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:00.904933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-f0131aa3-1d42-4896-a1c0-5b3580981c61 None None] MFA Rules not processed for user `b2d2a866714e47f3ab7e4d47bdae1a9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:00.906175 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f0131aa3-1d42-4896-a1c0-5b3580981c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:01.001588 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1047/2100] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:27:01.014430 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbd6fc-2d2c-42f6-8127-e2c33d72bb64 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:01.016294 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-cfcbd6fc-2d2c-42f6-8127-e2c33d72bb64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:01.095009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbd6fc-2d2c-42f6-8127-e2c33d72bb64 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:01.099897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d6cc4-60e1-4eaa-ab6d-870d7a98589c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:01.101247 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d6cc4-60e1-4eaa-ab6d-870d7a98589c service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:01.101768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d6cc4-60e1-4eaa-ab6d-870d7a98589c service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:01.104382 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d6cc4-60e1-4eaa-ab6d-870d7a98589c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:01.182658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d6cc4-60e1-4eaa-ab6d-870d7a98589c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:01.185688 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d6cc4-60e1-4eaa-ab6d-870d7a98589c service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:01.217658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1054/2101] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:05.737525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92e584-aac0-4dfd-ad23-f993450ef61d None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:05.737975 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92e584-aac0-4dfd-ad23-f993450ef61d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:05.738068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92e584-aac0-4dfd-ad23-f993450ef61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:05.772375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ae92e584-aac0-4dfd-ad23-f993450ef61d None None] MFA Rules not processed for user `d7ac3fe714fb48459a3524d4a458de62`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:05.773868 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ae92e584-aac0-4dfd-ad23-f993450ef61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:05.879279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1048/2102] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:05.887852 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d6cc4-60e1-4eaa-ab6d-870d7a98589c service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:05.889547 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-607d6cc4-60e1-4eaa-ab6d-870d7a98589c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:05.896655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:05.967163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d6cc4-60e1-4eaa-ab6d-870d7a98589c service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:05.972914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d69b3b-432e-4d64-86d6-97a5a935a53e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:05.973515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d69b3b-432e-4d64-86d6-97a5a935a53e service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:05.973823 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d69b3b-432e-4d64-86d6-97a5a935a53e service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:05.974704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d69b3b-432e-4d64-86d6-97a5a935a53e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:06.032670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d69b3b-432e-4d64-86d6-97a5a935a53e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:06.034180 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d69b3b-432e-4d64-86d6-97a5a935a53e service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:06.044243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1055/2103] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:06.735929 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92e584-aac0-4dfd-ad23-f993450ef61d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:06.752131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92e584-aac0-4dfd-ad23-f993450ef61d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:06.759444 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6f7c2-d9c1-444a-acde-795cc91c3383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:06.762284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6f7c2-d9c1-444a-acde-795cc91c3383 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:06.762836 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6f7c2-d9c1-444a-acde-795cc91c3383 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:06.764088 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6f7c2-d9c1-444a-acde-795cc91c3383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:06.822755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:06.844220 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6f7c2-d9c1-444a-acde-795cc91c3383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:06.846069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6f7c2-d9c1-444a-acde-795cc91c3383 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:06.861385 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1049/2104] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:09.178040 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d69b3b-432e-4d64-86d6-97a5a935a53e service nova] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:09.189625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d69b3b-432e-4d64-86d6-97a5a935a53e service nova] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:09.194761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97af0d0-0eea-43b3-8621-71f9edc2a714 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:09.195447 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97af0d0-0eea-43b3-8621-71f9edc2a714 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:09.195728 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97af0d0-0eea-43b3-8621-71f9edc2a714 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:09.196179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97af0d0-0eea-43b3-8621-71f9edc2a714 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:09.207906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97af0d0-0eea-43b3-8621-71f9edc2a714 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:09.209473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97af0d0-0eea-43b3-8621-71f9edc2a714 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:09.222413 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1056/2105] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:10.441738 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6f7c2-d9c1-444a-acde-795cc91c3383 service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:10.443245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-04b6f7c2-d9c1-444a-acde-795cc91c3383 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:10.529407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6f7c2-d9c1-444a-acde-795cc91c3383 service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:10.535244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c7820-a8ee-480c-b9e8-e01b2cd6d573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:10.536107 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c7820-a8ee-480c-b9e8-e01b2cd6d573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:10.536551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c7820-a8ee-480c-b9e8-e01b2cd6d573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:10.537334 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c7820-a8ee-480c-b9e8-e01b2cd6d573 admin admin] PATH_INFO: `/v3/users/ad2d61e33ade485784a3cb304ac02918` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:10.538765 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c7820-a8ee-480c-b9e8-e01b2cd6d573 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad2d61e33ade485784a3cb304ac02918)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:10.541738 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c7820-a8ee-480c-b9e8-e01b2cd6d573 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:10.771496 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d06c7820-a8ee-480c-b9e8-e01b2cd6d573 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad2d61e33ade485784a3cb304ac02918'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:10.825795 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d06c7820-a8ee-480c-b9e8-e01b2cd6d573 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad2d61e33ade485784a3cb304ac02918'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:10.826702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d06c7820-a8ee-480c-b9e8-e01b2cd6d573 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad2d61e33ade485784a3cb304ac02918'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:10.841684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d06c7820-a8ee-480c-b9e8-e01b2cd6d573 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad2d61e33ade485784a3cb304ac02918'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:10.842389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d06c7820-a8ee-480c-b9e8-e01b2cd6d573 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad2d61e33ade485784a3cb304ac02918'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:10.858074 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1050/2106] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:10 2022] DELETE /identity/v3/users/ad2d61e33ade485784a3cb304ac02918 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:10.862480 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c2106-9a44-42b1-b5f6-06a5eae5a1a5 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:10.863050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c2106-9a44-42b1-b5f6-06a5eae5a1a5 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:10.863207 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c2106-9a44-42b1-b5f6-06a5eae5a1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:10.883488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-b59c2106-9a44-42b1-b5f6-06a5eae5a1a5 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:10.887857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-b59c2106-9a44-42b1-b5f6-06a5eae5a1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:10.948888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1057/2107] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:10.958636 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c7820-a8ee-480c-b9e8-e01b2cd6d573 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:10.960047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d06c7820-a8ee-480c-b9e8-e01b2cd6d573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:11.027484 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c7820-a8ee-480c-b9e8-e01b2cd6d573 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:11.037315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a75a8b-66e4-4cc8-bfc1-44edce53b2e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:11.037315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a75a8b-66e4-4cc8-bfc1-44edce53b2e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:11.037315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a75a8b-66e4-4cc8-bfc1-44edce53b2e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:11.037315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a75a8b-66e4-4cc8-bfc1-44edce53b2e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:11.048630 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a75a8b-66e4-4cc8-bfc1-44edce53b2e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:11.050224 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a75a8b-66e4-4cc8-bfc1-44edce53b2e9 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:11.056073 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:11.063133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1051/2108] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:11.540709 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c2106-9a44-42b1-b5f6-06a5eae5a1a5 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:11.542195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-b59c2106-9a44-42b1-b5f6-06a5eae5a1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:11.554545 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c2106-9a44-42b1-b5f6-06a5eae5a1a5 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:11.559803 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436804d1-a254-46f9-9e21-4a4958ec53d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:11.560443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436804d1-a254-46f9-9e21-4a4958ec53d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:11.560722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436804d1-a254-46f9-9e21-4a4958ec53d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:11.560994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436804d1-a254-46f9-9e21-4a4958ec53d6 admin admin] PATH_INFO: `/v3/projects/e30b6e1da3f84b46b24305d175e41762` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:11.562134 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436804d1-a254-46f9-9e21-4a4958ec53d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e30b6e1da3f84b46b24305d175e41762)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:11.564109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436804d1-a254-46f9-9e21-4a4958ec53d6 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:11.587269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:11.675768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-436804d1-a254-46f9-9e21-4a4958ec53d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e30b6e1da3f84b46b24305d175e41762', 'request_id': 'req-436804d1-a254-46f9-9e21-4a4958ec53d6'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:11.687050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-436804d1-a254-46f9-9e21-4a4958ec53d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e30b6e1da3f84b46b24305d175e41762', 'request_id': 'req-436804d1-a254-46f9-9e21-4a4958ec53d6'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:11.766567 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-436804d1-a254-46f9-9e21-4a4958ec53d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e30b6e1da3f84b46b24305d175e41762', 'request_id': 'req-436804d1-a254-46f9-9e21-4a4958ec53d6'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:11.779765 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-436804d1-a254-46f9-9e21-4a4958ec53d6 admin admin] The token cache is being invalidate because project e30b6e1da3f84b46b24305d175e41762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:11.780314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-436804d1-a254-46f9-9e21-4a4958ec53d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:11.781192 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-436804d1-a254-46f9-9e21-4a4958ec53d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:11.782751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1058/2109] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:11 2022] DELETE /identity/v3/projects/e30b6e1da3f84b46b24305d175e41762 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:11.786849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a75a8b-66e4-4cc8-bfc1-44edce53b2e9 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:11.799452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a75a8b-66e4-4cc8-bfc1-44edce53b2e9 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:11.893318 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb156e-2007-42de-bef5-3c3f9f0b29d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:11.893958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb156e-2007-42de-bef5-3c3f9f0b29d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:11.894239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb156e-2007-42de-bef5-3c3f9f0b29d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:11.894512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb156e-2007-42de-bef5-3c3f9f0b29d5 admin admin] PATH_INFO: `/v3/users/d79d52ba84564ca7895f2da84e4238db` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:11.895698 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb156e-2007-42de-bef5-3c3f9f0b29d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d79d52ba84564ca7895f2da84e4238db)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:11.898744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb156e-2007-42de-bef5-3c3f9f0b29d5 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:12.080101 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-60cb156e-2007-42de-bef5-3c3f9f0b29d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd79d52ba84564ca7895f2da84e4238db'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:12.144294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-60cb156e-2007-42de-bef5-3c3f9f0b29d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd79d52ba84564ca7895f2da84e4238db'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:12.144912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-60cb156e-2007-42de-bef5-3c3f9f0b29d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd79d52ba84564ca7895f2da84e4238db'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:12.158694 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-60cb156e-2007-42de-bef5-3c3f9f0b29d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd79d52ba84564ca7895f2da84e4238db'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:12.159422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-60cb156e-2007-42de-bef5-3c3f9f0b29d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd79d52ba84564ca7895f2da84e4238db'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:12.174323 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1052/2110] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:11 2022] DELETE /identity/v3/users/d79d52ba84564ca7895f2da84e4238db => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:27:12.616473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436804d1-a254-46f9-9e21-4a4958ec53d6 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:12.619487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-436804d1-a254-46f9-9e21-4a4958ec53d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:12.639297 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436804d1-a254-46f9-9e21-4a4958ec53d6 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:12.702631 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c83ac4-a32a-43ba-835a-4a6e345dbf5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:12.703394 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c83ac4-a32a-43ba-835a-4a6e345dbf5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:12.703683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c83ac4-a32a-43ba-835a-4a6e345dbf5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:12.703960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c83ac4-a32a-43ba-835a-4a6e345dbf5a admin admin] PATH_INFO: `/v3/projects/6c6ca5b7ad5749cfb6901c6f043f6f48` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:12.705110 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c83ac4-a32a-43ba-835a-4a6e345dbf5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c6ca5b7ad5749cfb6901c6f043f6f48)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:12.707081 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c83ac4-a32a-43ba-835a-4a6e345dbf5a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:12.863959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-19c83ac4-a32a-43ba-835a-4a6e345dbf5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c6ca5b7ad5749cfb6901c6f043f6f48', 'request_id': 'req-19c83ac4-a32a-43ba-835a-4a6e345dbf5a'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:12.872536 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-19c83ac4-a32a-43ba-835a-4a6e345dbf5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c6ca5b7ad5749cfb6901c6f043f6f48', 'request_id': 'req-19c83ac4-a32a-43ba-835a-4a6e345dbf5a'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:12.940534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-19c83ac4-a32a-43ba-835a-4a6e345dbf5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c6ca5b7ad5749cfb6901c6f043f6f48', 'request_id': 'req-19c83ac4-a32a-43ba-835a-4a6e345dbf5a'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:12.953616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-19c83ac4-a32a-43ba-835a-4a6e345dbf5a admin admin] The token cache is being invalidate because project 6c6ca5b7ad5749cfb6901c6f043f6f48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:12.954169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-19c83ac4-a32a-43ba-835a-4a6e345dbf5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:12.954842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-19c83ac4-a32a-43ba-835a-4a6e345dbf5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:12.956406 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1059/2111] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:12 2022] DELETE /identity/v3/projects/6c6ca5b7ad5749cfb6901c6f043f6f48 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:12.964518 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66474c27-0735-4574-8e84-f4a101c8982d None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:12.965238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66474c27-0735-4574-8e84-f4a101c8982d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:12.965238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66474c27-0735-4574-8e84-f4a101c8982d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:12.987234 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-66474c27-0735-4574-8e84-f4a101c8982d None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:12.988564 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-66474c27-0735-4574-8e84-f4a101c8982d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:13.054186 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1053/2112] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:13.071877 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4841d3-9071-477b-b8ec-4bebe1fccc9c None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:13.072188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4841d3-9071-477b-b8ec-4bebe1fccc9c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:13.072462 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4841d3-9071-477b-b8ec-4bebe1fccc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:13.093610 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-cb4841d3-9071-477b-b8ec-4bebe1fccc9c None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:13.094853 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-cb4841d3-9071-477b-b8ec-4bebe1fccc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:13.103926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1060/2113] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:13.109234 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66474c27-0735-4574-8e84-f4a101c8982d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:13.119149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66474c27-0735-4574-8e84-f4a101c8982d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:13.123734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ad2b2-c633-4d41-a000-570312a634f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:13.124336 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ad2b2-c633-4d41-a000-570312a634f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:13.124619 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ad2b2-c633-4d41-a000-570312a634f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:13.124892 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ad2b2-c633-4d41-a000-570312a634f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:13.125542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ad2b2-c633-4d41-a000-570312a634f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:27:13.125846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ad2b2-c633-4d41-a000-570312a634f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:13.127476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ad2b2-c633-4d41-a000-570312a634f5 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:13.139632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1054/2114] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:27:13.143938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4841d3-9071-477b-b8ec-4bebe1fccc9c None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:13.153656 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4841d3-9071-477b-b8ec-4bebe1fccc9c None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:13.158455 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279eba2-01f2-459a-8876-2bf29339838a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:13.159094 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279eba2-01f2-459a-8876-2bf29339838a admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:13.159378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279eba2-01f2-459a-8876-2bf29339838a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:13.159652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279eba2-01f2-459a-8876-2bf29339838a admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:13.160293 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279eba2-01f2-459a-8876-2bf29339838a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:13.162196 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279eba2-01f2-459a-8876-2bf29339838a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:13.257711 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1061/2115] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:13 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:13.258842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ad2b2-c633-4d41-a000-570312a634f5 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:13.269616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ad2b2-c633-4d41-a000-570312a634f5 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:13.275607 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:13.331814 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c770bb-01fb-429b-bd88-86130e27f9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:13.332433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c770bb-01fb-429b-bd88-86130e27f9a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:13.332710 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c770bb-01fb-429b-bd88-86130e27f9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:13.333040 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c770bb-01fb-429b-bd88-86130e27f9a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:13.333624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c770bb-01fb-429b-bd88-86130e27f9a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:13.335614 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c770bb-01fb-429b-bd88-86130e27f9a7 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:13.446528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1055/2116] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:13 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:13.447761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279eba2-01f2-459a-8876-2bf29339838a admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:13.461379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279eba2-01f2-459a-8876-2bf29339838a admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:13.466048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81002656-06d7-41e2-b4be-33b9698bd34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:13.466656 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81002656-06d7-41e2-b4be-33b9698bd34a admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:13.466972 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81002656-06d7-41e2-b4be-33b9698bd34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:13.467244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81002656-06d7-41e2-b4be-33b9698bd34a admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:13.467941 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81002656-06d7-41e2-b4be-33b9698bd34a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:13.469510 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81002656-06d7-41e2-b4be-33b9698bd34a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:13.475842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:13.481027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1062/2117] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:27:13.485251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c770bb-01fb-429b-bd88-86130e27f9a7 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:13.496467 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c770bb-01fb-429b-bd88-86130e27f9a7 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:13.503895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26371d1-3f8a-475e-a919-d9cd9b278228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:13.504517 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26371d1-3f8a-475e-a919-d9cd9b278228 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:13.504808 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26371d1-3f8a-475e-a919-d9cd9b278228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:13.505110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26371d1-3f8a-475e-a919-d9cd9b278228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:13.505758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26371d1-3f8a-475e-a919-d9cd9b278228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:13.507403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26371d1-3f8a-475e-a919-d9cd9b278228 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:13.519620 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1056/2118] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:13.524563 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81002656-06d7-41e2-b4be-33b9698bd34a admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:13.534361 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81002656-06d7-41e2-b4be-33b9698bd34a admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:13.538914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fdaaf-0832-4a17-892b-8b2db6e63cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:13.539706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fdaaf-0832-4a17-892b-8b2db6e63cc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:13.543415 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fdaaf-0832-4a17-892b-8b2db6e63cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:13.543415 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fdaaf-0832-4a17-892b-8b2db6e63cc0 admin admin] PATH_INFO: `/v3/projects/616e1961dc1443f49be93b4d6a3100a3/users/1630cca00b5e453093d6bb568d2ba213/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:13.556161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426fdaaf-0832-4a17-892b-8b2db6e63cc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=616e1961dc1443f49be93b4d6a3100a3, user_id=1630cca00b5e453093d6bb568d2ba213, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:13.561277 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426fdaaf-0832-4a17-892b-8b2db6e63cc0 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:13.642133 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1063/2119] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:13 2022] PUT /identity/v3/projects/616e1961dc1443f49be93b4d6a3100a3/users/1630cca00b5e453093d6bb568d2ba213/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:13.651159 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6f897-0465-4f37-8de0-58b9be0ad390 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:13.651449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6f897-0465-4f37-8de0-58b9be0ad390 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:13.651729 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6f897-0465-4f37-8de0-58b9be0ad390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:13.682066 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ada6f897-0465-4f37-8de0-58b9be0ad390 None None] MFA Rules not processed for user `1630cca00b5e453093d6bb568d2ba213`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:13.683409 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ada6f897-0465-4f37-8de0-58b9be0ad390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:13.754077 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1057/2120] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:13.802512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fdaaf-0832-4a17-892b-8b2db6e63cc0 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:13.804522 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-426fdaaf-0832-4a17-892b-8b2db6e63cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:13.872563 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fdaaf-0832-4a17-892b-8b2db6e63cc0 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:13.876439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c9541-448a-4364-9ff8-22b924746bd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:13.877109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c9541-448a-4364-9ff8-22b924746bd3 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:13.877393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c9541-448a-4364-9ff8-22b924746bd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:13.877673 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c9541-448a-4364-9ff8-22b924746bd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:13.888393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c9541-448a-4364-9ff8-22b924746bd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:13.889834 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c9541-448a-4364-9ff8-22b924746bd3 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:13.900510 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1064/2121] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:14.964884 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6f897-0465-4f37-8de0-58b9be0ad390 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:14.966581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ada6f897-0465-4f37-8de0-58b9be0ad390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:15.073546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6f897-0465-4f37-8de0-58b9be0ad390 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:15.079774 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750bb29-799a-47de-9d80-a7c84cf8a03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:15.082184 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750bb29-799a-47de-9d80-a7c84cf8a03d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:15.082184 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750bb29-799a-47de-9d80-a7c84cf8a03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:15.082184 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750bb29-799a-47de-9d80-a7c84cf8a03d admin admin] PATH_INFO: `/v3/users/b0185e15d06f4a7c981dbdac309fcddd` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:15.082184 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750bb29-799a-47de-9d80-a7c84cf8a03d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0185e15d06f4a7c981dbdac309fcddd)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:15.084446 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750bb29-799a-47de-9d80-a7c84cf8a03d admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:15.104150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:15.295210 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4750bb29-799a-47de-9d80-a7c84cf8a03d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0185e15d06f4a7c981dbdac309fcddd'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:15.369282 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4750bb29-799a-47de-9d80-a7c84cf8a03d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0185e15d06f4a7c981dbdac309fcddd'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:15.370044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4750bb29-799a-47de-9d80-a7c84cf8a03d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0185e15d06f4a7c981dbdac309fcddd'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:15.384879 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4750bb29-799a-47de-9d80-a7c84cf8a03d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0185e15d06f4a7c981dbdac309fcddd'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:15.385733 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4750bb29-799a-47de-9d80-a7c84cf8a03d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0185e15d06f4a7c981dbdac309fcddd'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:15.400506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1058/2122] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:14 2022] DELETE /identity/v3/users/b0185e15d06f4a7c981dbdac309fcddd => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:15.405316 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb454a-afc2-4530-a8f7-1ffb0a16a5ad None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:15.405832 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb454a-afc2-4530-a8f7-1ffb0a16a5ad None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:15.405928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb454a-afc2-4530-a8f7-1ffb0a16a5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:15.434366 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-0cfb454a-afc2-4530-a8f7-1ffb0a16a5ad None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:15.435879 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-0cfb454a-afc2-4530-a8f7-1ffb0a16a5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:15.519536 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1065/2123] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:27:15.540237 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750bb29-799a-47de-9d80-a7c84cf8a03d admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:15.542004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4750bb29-799a-47de-9d80-a7c84cf8a03d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:15.635012 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750bb29-799a-47de-9d80-a7c84cf8a03d admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:15.640818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1890c82-1514-44c4-a3de-51626eff79f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:15.641851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1890c82-1514-44c4-a3de-51626eff79f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:15.642305 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1890c82-1514-44c4-a3de-51626eff79f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:15.642717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1890c82-1514-44c4-a3de-51626eff79f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:15.655390 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1890c82-1514-44c4-a3de-51626eff79f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:15.656940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1890c82-1514-44c4-a3de-51626eff79f3 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:15.669932 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1059/2124] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:27:16.471294 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb454a-afc2-4530-a8f7-1ffb0a16a5ad None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:16.472919 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-0cfb454a-afc2-4530-a8f7-1ffb0a16a5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:16.486227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb454a-afc2-4530-a8f7-1ffb0a16a5ad None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:16.494297 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:16.497698 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:16.497988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:16.501173 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b admin admin] PATH_INFO: `/v3/projects/3f96cd76884c4f72a7647ca56fd1c761` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:16.501173 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f96cd76884c4f72a7647ca56fd1c761)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:16.502171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:16.629952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f96cd76884c4f72a7647ca56fd1c761', 'request_id': 'req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:16.638567 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f96cd76884c4f72a7647ca56fd1c761', 'request_id': 'req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:16.728225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f96cd76884c4f72a7647ca56fd1c761', 'request_id': 'req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:16.742826 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b admin admin] The token cache is being invalidate because project 3f96cd76884c4f72a7647ca56fd1c761 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:16.747786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:16.747786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:16.749022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1066/2125] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:16 2022] DELETE /identity/v3/projects/3f96cd76884c4f72a7647ca56fd1c761 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:16.753362 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1890c82-1514-44c4-a3de-51626eff79f3 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:16.766072 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1890c82-1514-44c4-a3de-51626eff79f3 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:16.848383 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112d975-4105-4085-88cc-8fb2a76061cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:16.848973 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112d975-4105-4085-88cc-8fb2a76061cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:16.849252 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112d975-4105-4085-88cc-8fb2a76061cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:16.849525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112d975-4105-4085-88cc-8fb2a76061cc admin admin] PATH_INFO: `/v3/users/6e8f8fcda8754323ba4217a6fcf61ef6` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:16.851074 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112d975-4105-4085-88cc-8fb2a76061cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8f8fcda8754323ba4217a6fcf61ef6)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:16.853075 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112d975-4105-4085-88cc-8fb2a76061cc admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:17.068294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:17.077659 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8112d975-4105-4085-88cc-8fb2a76061cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8f8fcda8754323ba4217a6fcf61ef6'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:17.207580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8112d975-4105-4085-88cc-8fb2a76061cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8f8fcda8754323ba4217a6fcf61ef6'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:17.208924 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8112d975-4105-4085-88cc-8fb2a76061cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8f8fcda8754323ba4217a6fcf61ef6'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:17.226625 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8112d975-4105-4085-88cc-8fb2a76061cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8f8fcda8754323ba4217a6fcf61ef6'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:17.231658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8112d975-4105-4085-88cc-8fb2a76061cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8f8fcda8754323ba4217a6fcf61ef6'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:17.254559 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1060/2126] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:16 2022] DELETE /identity/v3/users/6e8f8fcda8754323ba4217a6fcf61ef6 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:27:17.739872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:17.741822 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:17.753474 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:17.764015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b76c6d-196f-4399-bdf7-c0ee93dc2f8b admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:17.855090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891aac5-715b-44a2-b978-c2c6bd052e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:17.856235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891aac5-715b-44a2-b978-c2c6bd052e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:17.856711 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891aac5-715b-44a2-b978-c2c6bd052e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:17.857170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891aac5-715b-44a2-b978-c2c6bd052e79 admin admin] PATH_INFO: `/v3/projects/4b919359e41f4c8c8e442376912e228a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:17.861927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8891aac5-715b-44a2-b978-c2c6bd052e79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b919359e41f4c8c8e442376912e228a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:17.864386 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8891aac5-715b-44a2-b978-c2c6bd052e79 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:17.969322 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-8891aac5-715b-44a2-b978-c2c6bd052e79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b919359e41f4c8c8e442376912e228a', 'request_id': 'req-8891aac5-715b-44a2-b978-c2c6bd052e79'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:17.976074 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-8891aac5-715b-44a2-b978-c2c6bd052e79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b919359e41f4c8c8e442376912e228a', 'request_id': 'req-8891aac5-715b-44a2-b978-c2c6bd052e79'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:18.043077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-8891aac5-715b-44a2-b978-c2c6bd052e79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b919359e41f4c8c8e442376912e228a', 'request_id': 'req-8891aac5-715b-44a2-b978-c2c6bd052e79'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:18.059538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-8891aac5-715b-44a2-b978-c2c6bd052e79 admin admin] The token cache is being invalidate because project 4b919359e41f4c8c8e442376912e228a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:18.059838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-8891aac5-715b-44a2-b978-c2c6bd052e79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:18.060528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-8891aac5-715b-44a2-b978-c2c6bd052e79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:18.062075 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1067/2127] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:17 2022] DELETE /identity/v3/projects/4b919359e41f4c8c8e442376912e228a => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:18.066155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112d975-4105-4085-88cc-8fb2a76061cc admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:18.067819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8112d975-4105-4085-88cc-8fb2a76061cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:18.082649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112d975-4105-4085-88cc-8fb2a76061cc admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:18.152109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af2e3f-a09c-48d4-99aa-3899c75b5f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:18.152982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af2e3f-a09c-48d4-99aa-3899c75b5f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:18.153369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af2e3f-a09c-48d4-99aa-3899c75b5f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:18.153830 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af2e3f-a09c-48d4-99aa-3899c75b5f60 admin admin] PATH_INFO: `/v3/users/b2d2a866714e47f3ab7e4d47bdae1a9b` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:18.156245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af2e3f-a09c-48d4-99aa-3899c75b5f60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2d2a866714e47f3ab7e4d47bdae1a9b)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:18.158721 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af2e3f-a09c-48d4-99aa-3899c75b5f60 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:18.404195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-45af2e3f-a09c-48d4-99aa-3899c75b5f60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2d2a866714e47f3ab7e4d47bdae1a9b'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:18.511887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-45af2e3f-a09c-48d4-99aa-3899c75b5f60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2d2a866714e47f3ab7e4d47bdae1a9b'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:18.512818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-45af2e3f-a09c-48d4-99aa-3899c75b5f60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2d2a866714e47f3ab7e4d47bdae1a9b'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:18.530185 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-45af2e3f-a09c-48d4-99aa-3899c75b5f60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2d2a866714e47f3ab7e4d47bdae1a9b'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:18.531040 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-45af2e3f-a09c-48d4-99aa-3899c75b5f60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2d2a866714e47f3ab7e4d47bdae1a9b'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:18.552720 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1061/2128] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:18 2022] DELETE /identity/v3/users/b2d2a866714e47f3ab7e4d47bdae1a9b => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:27:19.203391 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891aac5-715b-44a2-b978-c2c6bd052e79 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:19.207001 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8891aac5-715b-44a2-b978-c2c6bd052e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:19.257493 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891aac5-715b-44a2-b978-c2c6bd052e79 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:19.368024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a0eff-a9dd-4230-a745-d353821da994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:19.369025 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a0eff-a9dd-4230-a745-d353821da994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:19.369742 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a0eff-a9dd-4230-a745-d353821da994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:19.370216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a0eff-a9dd-4230-a745-d353821da994 admin admin] PATH_INFO: `/v3/projects/c01d926e8e2d4b06a21471e3413ed478` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:19.373005 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a0eff-a9dd-4230-a745-d353821da994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c01d926e8e2d4b06a21471e3413ed478)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:19.377507 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a0eff-a9dd-4230-a745-d353821da994 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:19.471248 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af2e3f-a09c-48d4-99aa-3899c75b5f60 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:19.472966 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-45af2e3f-a09c-48d4-99aa-3899c75b5f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:19.491176 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af2e3f-a09c-48d4-99aa-3899c75b5f60 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:19.499749 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba537ab-7b84-4381-94e6-03bcb519b1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:19.500406 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba537ab-7b84-4381-94e6-03bcb519b1f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:19.500687 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba537ab-7b84-4381-94e6-03bcb519b1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:19.500967 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba537ab-7b84-4381-94e6-03bcb519b1f7 admin admin] PATH_INFO: `/v3/users/1630cca00b5e453093d6bb568d2ba213` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:19.502102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba537ab-7b84-4381-94e6-03bcb519b1f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1630cca00b5e453093d6bb568d2ba213)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:19.504108 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba537ab-7b84-4381-94e6-03bcb519b1f7 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:19.567281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:19.580737 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f60a0eff-a9dd-4230-a745-d353821da994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c01d926e8e2d4b06a21471e3413ed478', 'request_id': 'req-f60a0eff-a9dd-4230-a745-d353821da994'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:19.591367 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f60a0eff-a9dd-4230-a745-d353821da994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c01d926e8e2d4b06a21471e3413ed478', 'request_id': 'req-f60a0eff-a9dd-4230-a745-d353821da994'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:19.723451 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f60a0eff-a9dd-4230-a745-d353821da994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c01d926e8e2d4b06a21471e3413ed478', 'request_id': 'req-f60a0eff-a9dd-4230-a745-d353821da994'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:19.727325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:19.739430 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f60a0eff-a9dd-4230-a745-d353821da994 admin admin] The token cache is being invalidate because project c01d926e8e2d4b06a21471e3413ed478 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:19.739729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f60a0eff-a9dd-4230-a745-d353821da994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:19.740401 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f60a0eff-a9dd-4230-a745-d353821da994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:19.740676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-2ba537ab-7b84-4381-94e6-03bcb519b1f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1630cca00b5e453093d6bb568d2ba213'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:19.743052 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1068/2129] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:19 2022] DELETE /identity/v3/projects/c01d926e8e2d4b06a21471e3413ed478 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:19.754310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7271015-e278-48bd-8071-85f7ba60c15b None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:19.762030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7271015-e278-48bd-8071-85f7ba60c15b None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:19.762030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7271015-e278-48bd-8071-85f7ba60c15b None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:19.762030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1069/2130] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 31 19:27:19 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:27:19.781727 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1fc70-3763-4501-a660-a6a0b7614605 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:19.782618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1fc70-3763-4501-a660-a6a0b7614605 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:19.790340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1fc70-3763-4501-a660-a6a0b7614605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:19.813313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-7fd1fc70-3763-4501-a660-a6a0b7614605 None None] MFA Rules not processed for user `eced13cc4030423abe63bb1527f6d35a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:19.814690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7fd1fc70-3763-4501-a660-a6a0b7614605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:19.848922 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-2ba537ab-7b84-4381-94e6-03bcb519b1f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1630cca00b5e453093d6bb568d2ba213'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:19.850584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-2ba537ab-7b84-4381-94e6-03bcb519b1f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1630cca00b5e453093d6bb568d2ba213'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:19.880220 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-2ba537ab-7b84-4381-94e6-03bcb519b1f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1630cca00b5e453093d6bb568d2ba213'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:19.881530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-2ba537ab-7b84-4381-94e6-03bcb519b1f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1630cca00b5e453093d6bb568d2ba213'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:19.903306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1062/2131] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:19 2022] DELETE /identity/v3/users/1630cca00b5e453093d6bb568d2ba213 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:27:19.904633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba537ab-7b84-4381-94e6-03bcb519b1f7 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:19.906226 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-2ba537ab-7b84-4381-94e6-03bcb519b1f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:19.914984 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1070/2132] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 31 19:27:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:19.916370 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fa778-7ae2-454d-b9c8-4adb3163cc36 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:19.917190 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fa778-7ae2-454d-b9c8-4adb3163cc36 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:19.917565 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fa778-7ae2-454d-b9c8-4adb3163cc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:19.953195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-171fa778-7ae2-454d-b9c8-4adb3163cc36 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:19.954413 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-171fa778-7ae2-454d-b9c8-4adb3163cc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:20.003871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba537ab-7b84-4381-94e6-03bcb519b1f7 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:20.009994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70971d-5e1e-4765-a34c-26a00a6ad223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:20.010674 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70971d-5e1e-4765-a34c-26a00a6ad223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:20.010981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70971d-5e1e-4765-a34c-26a00a6ad223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:20.011302 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70971d-5e1e-4765-a34c-26a00a6ad223 admin admin] PATH_INFO: `/v3/users/7aaa082f2b754cf5aa941586361c1877` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:20.016998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70971d-5e1e-4765-a34c-26a00a6ad223 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aaa082f2b754cf5aa941586361c1877)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:20.016998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70971d-5e1e-4765-a34c-26a00a6ad223 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:20.027180 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1071/2133] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:20.027640 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fa778-7ae2-454d-b9c8-4adb3163cc36 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:20.029001 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-171fa778-7ae2-454d-b9c8-4adb3163cc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:20.109415 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fa778-7ae2-454d-b9c8-4adb3163cc36 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:20.114680 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e763d-e6fc-4ea9-b64e-e15272893375 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:20.115408 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e763d-e6fc-4ea9-b64e-e15272893375 service placement] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:20.115694 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e763d-e6fc-4ea9-b64e-e15272893375 service placement] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:20.118150 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e763d-e6fc-4ea9-b64e-e15272893375 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:20.126505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e763d-e6fc-4ea9-b64e-e15272893375 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:20.128424 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e763d-e6fc-4ea9-b64e-e15272893375 service placement] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:20.140564 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1072/2134] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:20.141653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e763d-e6fc-4ea9-b64e-e15272893375 service placement] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:20.143004 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-fe6e763d-e6fc-4ea9-b64e-e15272893375 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:20.184733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:20.206807 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e763d-e6fc-4ea9-b64e-e15272893375 service placement] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:20.210514 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd7a8a-44e6-4245-bcf6-b9f2a1f9a2fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:20.211302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd7a8a-44e6-4245-bcf6-b9f2a1f9a2fc service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:20.211578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd7a8a-44e6-4245-bcf6-b9f2a1f9a2fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:20.211849 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd7a8a-44e6-4245-bcf6-b9f2a1f9a2fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:20.221718 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd7a8a-44e6-4245-bcf6-b9f2a1f9a2fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:20.223200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd7a8a-44e6-4245-bcf6-b9f2a1f9a2fc service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:20.233598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1073/2135] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:20.301845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4d70971d-5e1e-4765-a34c-26a00a6ad223 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aaa082f2b754cf5aa941586361c1877'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.418479 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4d70971d-5e1e-4765-a34c-26a00a6ad223 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aaa082f2b754cf5aa941586361c1877'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.419484 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4d70971d-5e1e-4765-a34c-26a00a6ad223 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aaa082f2b754cf5aa941586361c1877'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.440559 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4d70971d-5e1e-4765-a34c-26a00a6ad223 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aaa082f2b754cf5aa941586361c1877'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.441401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4d70971d-5e1e-4765-a34c-26a00a6ad223 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aaa082f2b754cf5aa941586361c1877'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.459043 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1063/2136] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:19 2022] DELETE /identity/v3/users/7aaa082f2b754cf5aa941586361c1877 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:20.773698 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd7a8a-44e6-4245-bcf6-b9f2a1f9a2fc service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:20.776732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e5bd7a8a-44e6-4245-bcf6-b9f2a1f9a2fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:20.801454 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd7a8a-44e6-4245-bcf6-b9f2a1f9a2fc service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:20.871515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef5e76-eb4a-4904-bd9b-a7c022b40929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:20.872148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef5e76-eb4a-4904-bd9b-a7c022b40929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:20.872601 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef5e76-eb4a-4904-bd9b-a7c022b40929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:20.872873 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef5e76-eb4a-4904-bd9b-a7c022b40929 admin admin] PATH_INFO: `/v3/projects/616e1961dc1443f49be93b4d6a3100a3` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:20.874067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef5e76-eb4a-4904-bd9b-a7c022b40929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=616e1961dc1443f49be93b4d6a3100a3)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:20.876038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef5e76-eb4a-4904-bd9b-a7c022b40929 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.015793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-89ef5e76-eb4a-4904-bd9b-a7c022b40929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '616e1961dc1443f49be93b4d6a3100a3', 'request_id': 'req-89ef5e76-eb4a-4904-bd9b-a7c022b40929'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:21.021594 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-89ef5e76-eb4a-4904-bd9b-a7c022b40929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '616e1961dc1443f49be93b4d6a3100a3', 'request_id': 'req-89ef5e76-eb4a-4904-bd9b-a7c022b40929'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:21.062110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70971d-5e1e-4765-a34c-26a00a6ad223 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.063845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4d70971d-5e1e-4765-a34c-26a00a6ad223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:21.079208 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70971d-5e1e-4765-a34c-26a00a6ad223 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.107259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:21.111502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-89ef5e76-eb4a-4904-bd9b-a7c022b40929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '616e1961dc1443f49be93b4d6a3100a3', 'request_id': 'req-89ef5e76-eb4a-4904-bd9b-a7c022b40929'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:21.124321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-89ef5e76-eb4a-4904-bd9b-a7c022b40929 admin admin] The token cache is being invalidate because project 616e1961dc1443f49be93b4d6a3100a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:21.133358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-89ef5e76-eb4a-4904-bd9b-a7c022b40929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:21.133358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-89ef5e76-eb4a-4904-bd9b-a7c022b40929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:21.133358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1074/2137] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:20 2022] DELETE /identity/v3/projects/616e1961dc1443f49be93b4d6a3100a3 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:21.133358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35b450-6adf-4478-91e6-835b0acb0837 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.133358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35b450-6adf-4478-91e6-835b0acb0837 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.133358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35b450-6adf-4478-91e6-835b0acb0837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.144743 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c211a8-6086-4f64-a6e7-3672114ac665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.145436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c211a8-6086-4f64-a6e7-3672114ac665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.145696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c211a8-6086-4f64-a6e7-3672114ac665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.145968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c211a8-6086-4f64-a6e7-3672114ac665 admin admin] PATH_INFO: `/v3/projects/fa8bb54679c74f039eee057c2b4a84f7` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.147329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c211a8-6086-4f64-a6e7-3672114ac665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa8bb54679c74f039eee057c2b4a84f7)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.149809 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c211a8-6086-4f64-a6e7-3672114ac665 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.156886 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-fd35b450-6adf-4478-91e6-835b0acb0837 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:21.158210 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-fd35b450-6adf-4478-91e6-835b0acb0837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:21.167778 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1075/2138] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:27:21.176618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853de7a-efb1-4c8e-9bff-9f16f7f3dc80 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.176977 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853de7a-efb1-4c8e-9bff-9f16f7f3dc80 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.177256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853de7a-efb1-4c8e-9bff-9f16f7f3dc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.198502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-5853de7a-efb1-4c8e-9bff-9f16f7f3dc80 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:21.199761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-5853de7a-efb1-4c8e-9bff-9f16f7f3dc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:21.208038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1076/2139] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:27:21.213143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853de7a-efb1-4c8e-9bff-9f16f7f3dc80 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.222673 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853de7a-efb1-4c8e-9bff-9f16f7f3dc80 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.228008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12bd8b-be49-4ec5-8cd9-64b0c33c9045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.228615 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12bd8b-be49-4ec5-8cd9-64b0c33c9045 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.228898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12bd8b-be49-4ec5-8cd9-64b0c33c9045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.229225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12bd8b-be49-4ec5-8cd9-64b0c33c9045 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.229886 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a12bd8b-be49-4ec5-8cd9-64b0c33c9045 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:27:21.230198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a12bd8b-be49-4ec5-8cd9-64b0c33c9045 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.231874 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a12bd8b-be49-4ec5-8cd9-64b0c33c9045 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.234377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:21.244103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1077/2140] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:27:21.248607 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12bd8b-be49-4ec5-8cd9-64b0c33c9045 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.259781 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12bd8b-be49-4ec5-8cd9-64b0c33c9045 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.271400 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-83c211a8-6086-4f64-a6e7-3672114ac665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa8bb54679c74f039eee057c2b4a84f7', 'request_id': 'req-83c211a8-6086-4f64-a6e7-3672114ac665'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:21.282086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-83c211a8-6086-4f64-a6e7-3672114ac665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa8bb54679c74f039eee057c2b4a84f7', 'request_id': 'req-83c211a8-6086-4f64-a6e7-3672114ac665'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:21.293563 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-83c211a8-6086-4f64-a6e7-3672114ac665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa8bb54679c74f039eee057c2b4a84f7', 'request_id': 'req-83c211a8-6086-4f64-a6e7-3672114ac665'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:21.320631 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905a19a-0251-46c8-adb3-81b6e63e3984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.321417 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905a19a-0251-46c8-adb3-81b6e63e3984 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.321726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905a19a-0251-46c8-adb3-81b6e63e3984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.322004 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905a19a-0251-46c8-adb3-81b6e63e3984 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.322671 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905a19a-0251-46c8-adb3-81b6e63e3984 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.324694 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905a19a-0251-46c8-adb3-81b6e63e3984 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.411887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-83c211a8-6086-4f64-a6e7-3672114ac665 admin admin] The token cache is being invalidate because project fa8bb54679c74f039eee057c2b4a84f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:21.412182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-83c211a8-6086-4f64-a6e7-3672114ac665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:21.412812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-83c211a8-6086-4f64-a6e7-3672114ac665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:21.415190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1064/2141] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:21 2022] DELETE /identity/v3/projects/fa8bb54679c74f039eee057c2b4a84f7 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:21.420059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745b594-6ece-4f33-adc2-7fc3a9d08d6f None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.420346 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745b594-6ece-4f33-adc2-7fc3a9d08d6f None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.420623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745b594-6ece-4f33-adc2-7fc3a9d08d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.446558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1078/2142] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:21 2022] POST /identity/v3/projects => generated 363 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:21.448001 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-b745b594-6ece-4f33-adc2-7fc3a9d08d6f None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:21.449406 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b745b594-6ece-4f33-adc2-7fc3a9d08d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:21.460232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905a19a-0251-46c8-adb3-81b6e63e3984 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.462287 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-b905a19a-0251-46c8-adb3-81b6e63e3984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:21.527154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1065/2143] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:27:21.536406 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b113b-6824-49fa-acfb-f7bf44d64011 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.537365 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b113b-6824-49fa-acfb-f7bf44d64011 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.537721 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b113b-6824-49fa-acfb-f7bf44d64011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.544834 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905a19a-0251-46c8-adb3-81b6e63e3984 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.552373 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0ec5c-8877-4ef1-a604-bca8757dd9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.553303 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0ec5c-8877-4ef1-a604-bca8757dd9ca admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.553657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0ec5c-8877-4ef1-a604-bca8757dd9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.554000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0ec5c-8877-4ef1-a604-bca8757dd9ca admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.556484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0ec5c-8877-4ef1-a604-bca8757dd9ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.558509 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0ec5c-8877-4ef1-a604-bca8757dd9ca admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.567406 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-8e0b113b-6824-49fa-acfb-f7bf44d64011 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:21.569191 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8e0b113b-6824-49fa-acfb-f7bf44d64011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:21.577622 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1066/2144] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:21.581842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b113b-6824-49fa-acfb-f7bf44d64011 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.592225 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b113b-6824-49fa-acfb-f7bf44d64011 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.596950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b077d-ac33-4dc0-90f5-5cf7883d610b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.597565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b077d-ac33-4dc0-90f5-5cf7883d610b admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.597830 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b077d-ac33-4dc0-90f5-5cf7883d610b admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.598100 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b077d-ac33-4dc0-90f5-5cf7883d610b admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.598738 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b077d-ac33-4dc0-90f5-5cf7883d610b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:27:21.599067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b077d-ac33-4dc0-90f5-5cf7883d610b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.600901 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b077d-ac33-4dc0-90f5-5cf7883d610b admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.617635 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1067/2145] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:27:21.623311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b077d-ac33-4dc0-90f5-5cf7883d610b admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.634797 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b077d-ac33-4dc0-90f5-5cf7883d610b admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.639050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1079/2146] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:21 2022] POST /identity/v3/users => generated 282 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:27:21.640109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd091c3e-7298-4337-bf92-7e05cecf8c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.640716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd091c3e-7298-4337-bf92-7e05cecf8c15 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.640997 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd091c3e-7298-4337-bf92-7e05cecf8c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.641267 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd091c3e-7298-4337-bf92-7e05cecf8c15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.641893 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd091c3e-7298-4337-bf92-7e05cecf8c15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.643476 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0ec5c-8877-4ef1-a604-bca8757dd9ca admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.643913 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd091c3e-7298-4337-bf92-7e05cecf8c15 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.654373 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0ec5c-8877-4ef1-a604-bca8757dd9ca admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.659970 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf51a7-0fe3-4528-8b31-f07156038701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.660834 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf51a7-0fe3-4528-8b31-f07156038701 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.661117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf51a7-0fe3-4528-8b31-f07156038701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.661389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf51a7-0fe3-4528-8b31-f07156038701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.662194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf51a7-0fe3-4528-8b31-f07156038701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.664061 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf51a7-0fe3-4528-8b31-f07156038701 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.674897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1080/2147] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:27:21.678803 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf51a7-0fe3-4528-8b31-f07156038701 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.689645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf51a7-0fe3-4528-8b31-f07156038701 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.694284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8263a-6a15-4843-a3ba-cacc14181ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.697927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8263a-6a15-4843-a3ba-cacc14181ccc admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.697927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8263a-6a15-4843-a3ba-cacc14181ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.697927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8263a-6a15-4843-a3ba-cacc14181ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.697927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8263a-6a15-4843-a3ba-cacc14181ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.698377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8263a-6a15-4843-a3ba-cacc14181ccc admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.713602 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1081/2148] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:27:21.714833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8263a-6a15-4843-a3ba-cacc14181ccc admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.725304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1068/2149] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:21 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:27:21.727766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8263a-6a15-4843-a3ba-cacc14181ccc admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.729561 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd091c3e-7298-4337-bf92-7e05cecf8c15 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.752067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:21.777590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:21.790400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca59bb-e1ce-4250-8b97-cb8568c01f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.791073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca59bb-e1ce-4250-8b97-cb8568c01f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.791353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca59bb-e1ce-4250-8b97-cb8568c01f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.791697 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca59bb-e1ce-4250-8b97-cb8568c01f69 admin admin] PATH_INFO: `/v3/projects/4326d5d73ed341e69cbf9ee14ad380c7/users/16b7ecd41b754880a769f3e1342bed6b/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.801881 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd091c3e-7298-4337-bf92-7e05cecf8c15 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.808102 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca59bb-e1ce-4250-8b97-cb8568c01f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4326d5d73ed341e69cbf9ee14ad380c7, user_id=16b7ecd41b754880a769f3e1342bed6b, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.808290 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488d161-c8f7-4a4b-aeb0-62d8bcf8bf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.808718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488d161-c8f7-4a4b-aeb0-62d8bcf8bf64 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.809001 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488d161-c8f7-4a4b-aeb0-62d8bcf8bf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.809276 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488d161-c8f7-4a4b-aeb0-62d8bcf8bf64 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.809914 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488d161-c8f7-4a4b-aeb0-62d8bcf8bf64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.810978 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca59bb-e1ce-4250-8b97-cb8568c01f69 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.811886 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488d161-c8f7-4a4b-aeb0-62d8bcf8bf64 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.900250 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1082/2150] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:21 2022] PUT /identity/v3/projects/4326d5d73ed341e69cbf9ee14ad380c7/users/16b7ecd41b754880a769f3e1342bed6b/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:21.910976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d1240-3fcf-494b-bde9-7dbd880300a2 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.911265 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d1240-3fcf-494b-bde9-7dbd880300a2 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.911732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d1240-3fcf-494b-bde9-7dbd880300a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.933902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1069/2151] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:21 2022] POST /identity/v3/users => generated 291 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:27:21.941460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488d161-c8f7-4a4b-aeb0-62d8bcf8bf64 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.945617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-f93d1240-3fcf-494b-bde9-7dbd880300a2 None None] MFA Rules not processed for user `16b7ecd41b754880a769f3e1342bed6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:21.947433 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f93d1240-3fcf-494b-bde9-7dbd880300a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:21.957548 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488d161-c8f7-4a4b-aeb0-62d8bcf8bf64 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:22.031705 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c062f15-a488-4286-a2b7-d63f40b9dd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:22.032335 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c062f15-a488-4286-a2b7-d63f40b9dd77 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:22.032738 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c062f15-a488-4286-a2b7-d63f40b9dd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:22.033038 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c062f15-a488-4286-a2b7-d63f40b9dd77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:22.033693 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c062f15-a488-4286-a2b7-d63f40b9dd77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:22.035487 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c062f15-a488-4286-a2b7-d63f40b9dd77 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:22.047474 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1083/2152] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:27:22.057338 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1070/2153] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:27:22.058424 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d1240-3fcf-494b-bde9-7dbd880300a2 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:22.068927 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c062f15-a488-4286-a2b7-d63f40b9dd77 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:22.073118 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8c062f15-a488-4286-a2b7-d63f40b9dd77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:22.077566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d1240-3fcf-494b-bde9-7dbd880300a2 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:22.084727 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030758b-8d88-4793-ac6d-97aec330278d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:22.085617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030758b-8d88-4793-ac6d-97aec330278d admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:22.088522 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030758b-8d88-4793-ac6d-97aec330278d admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:22.088816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030758b-8d88-4793-ac6d-97aec330278d admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:22.090527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030758b-8d88-4793-ac6d-97aec330278d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:22.093619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030758b-8d88-4793-ac6d-97aec330278d admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:22.110808 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1084/2154] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:22.114909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030758b-8d88-4793-ac6d-97aec330278d admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:22.126179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030758b-8d88-4793-ac6d-97aec330278d admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:22.131823 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b1f38-2c8b-4928-99d8-44f3b0717852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:22.135581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b1f38-2c8b-4928-99d8-44f3b0717852 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:22.135581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b1f38-2c8b-4928-99d8-44f3b0717852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:22.135581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b1f38-2c8b-4928-99d8-44f3b0717852 admin admin] PATH_INFO: `/v3/projects/d63f4d20e3a74f189d2d729a072152f3/users/a36ed4f9eb644f73874080e951734b41/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:22.152522 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b1f38-2c8b-4928-99d8-44f3b0717852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d63f4d20e3a74f189d2d729a072152f3, user_id=a36ed4f9eb644f73874080e951734b41, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:22.156201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b1f38-2c8b-4928-99d8-44f3b0717852 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:22.165255 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c062f15-a488-4286-a2b7-d63f40b9dd77 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:22.169162 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f734876-b559-4dcf-838c-fe2111b97e5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:22.169782 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f734876-b559-4dcf-838c-fe2111b97e5d service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:22.170067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f734876-b559-4dcf-838c-fe2111b97e5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:22.170343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f734876-b559-4dcf-838c-fe2111b97e5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:22.179479 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f734876-b559-4dcf-838c-fe2111b97e5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:22.181502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f734876-b559-4dcf-838c-fe2111b97e5d service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:22.191511 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1071/2155] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:22.258766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1085/2156] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:22 2022] PUT /identity/v3/projects/d63f4d20e3a74f189d2d729a072152f3/users/a36ed4f9eb644f73874080e951734b41/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:27:22.274461 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7202716-0be8-4873-a28b-16aa5c364803 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:22.274461 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7202716-0be8-4873-a28b-16aa5c364803 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:22.274461 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7202716-0be8-4873-a28b-16aa5c364803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:22.306816 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-e7202716-0be8-4873-a28b-16aa5c364803 None None] MFA Rules not processed for user `a36ed4f9eb644f73874080e951734b41`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:22.308333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e7202716-0be8-4873-a28b-16aa5c364803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:22.345358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:22.379886 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1072/2157] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:27:22.384680 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b1f38-2c8b-4928-99d8-44f3b0717852 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:22.455488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b1f38-2c8b-4928-99d8-44f3b0717852 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:22.460453 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043dda95-e2e5-48dc-8470-1d0744d8a9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:22.461116 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043dda95-e2e5-48dc-8470-1d0744d8a9d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:22.461399 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043dda95-e2e5-48dc-8470-1d0744d8a9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:22.461820 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043dda95-e2e5-48dc-8470-1d0744d8a9d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:22.462492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043dda95-e2e5-48dc-8470-1d0744d8a9d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:22.464451 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043dda95-e2e5-48dc-8470-1d0744d8a9d5 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:22.561752 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1086/2158] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:22 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:22.565341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7202716-0be8-4873-a28b-16aa5c364803 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:22.584238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7202716-0be8-4873-a28b-16aa5c364803 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:22.670202 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d523f-2c19-42c5-8f31-a129a4f08024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:22.671504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d523f-2c19-42c5-8f31-a129a4f08024 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:22.671794 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d523f-2c19-42c5-8f31-a129a4f08024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:22.672074 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d523f-2c19-42c5-8f31-a129a4f08024 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:22.673121 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d523f-2c19-42c5-8f31-a129a4f08024 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:22.675314 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d523f-2c19-42c5-8f31-a129a4f08024 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:22.781627 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1073/2159] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:22 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:22.787074 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043dda95-e2e5-48dc-8470-1d0744d8a9d5 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:22.796570 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043dda95-e2e5-48dc-8470-1d0744d8a9d5 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:22.801498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b689e-2a59-4262-829a-3daea848e62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:22.802110 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b689e-2a59-4262-829a-3daea848e62b admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:22.802396 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b689e-2a59-4262-829a-3daea848e62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:22.802670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b689e-2a59-4262-829a-3daea848e62b admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:22.803377 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b689e-2a59-4262-829a-3daea848e62b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:22.805132 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b689e-2a59-4262-829a-3daea848e62b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:22.817784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:22.825713 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1087/2160] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:22.830005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d523f-2c19-42c5-8f31-a129a4f08024 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:22.841017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d523f-2c19-42c5-8f31-a129a4f08024 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:22.845765 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a021e-8d90-4d5e-9397-dbee0f0008fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:22.846498 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a021e-8d90-4d5e-9397-dbee0f0008fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:22.846782 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a021e-8d90-4d5e-9397-dbee0f0008fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:22.847096 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a021e-8d90-4d5e-9397-dbee0f0008fa admin admin] PATH_INFO: `/v3/projects/895ce7c59fd34d31af3bfec5f7546743/users/149fdf407b624d90b476fca9bcfcec0f/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:22.864890 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a021e-8d90-4d5e-9397-dbee0f0008fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=895ce7c59fd34d31af3bfec5f7546743, user_id=149fdf407b624d90b476fca9bcfcec0f, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:22.867731 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a021e-8d90-4d5e-9397-dbee0f0008fa admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:22.984311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1074/2161] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:22 2022] PUT /identity/v3/projects/895ce7c59fd34d31af3bfec5f7546743/users/149fdf407b624d90b476fca9bcfcec0f/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:22.988233 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b689e-2a59-4262-829a-3daea848e62b admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:22.999197 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b689e-2a59-4262-829a-3daea848e62b admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:23.060301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fcb68-e608-49fb-9f30-871ff700613a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:23.060842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fcb68-e608-49fb-9f30-871ff700613a admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:23.061113 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fcb68-e608-49fb-9f30-871ff700613a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:23.061414 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fcb68-e608-49fb-9f30-871ff700613a admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:23.062048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fcb68-e608-49fb-9f30-871ff700613a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:23.063747 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fcb68-e608-49fb-9f30-871ff700613a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:23.073822 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1088/2162] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:23.078056 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a021e-8d90-4d5e-9397-dbee0f0008fa admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:23.089574 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a021e-8d90-4d5e-9397-dbee0f0008fa admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:23.094326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450c496-6f19-4c54-913a-ae03c6ed399c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:23.095026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450c496-6f19-4c54-913a-ae03c6ed399c admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:23.095360 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450c496-6f19-4c54-913a-ae03c6ed399c admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:23.095637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450c496-6f19-4c54-913a-ae03c6ed399c admin admin] PATH_INFO: `/v3/projects/895ce7c59fd34d31af3bfec5f7546743/users/149fdf407b624d90b476fca9bcfcec0f/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:23.097722 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450c496-6f19-4c54-913a-ae03c6ed399c admin admin] RBAC: Authorizing `identity:create_grant(project_id=895ce7c59fd34d31af3bfec5f7546743, user_id=149fdf407b624d90b476fca9bcfcec0f, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:23.100661 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450c496-6f19-4c54-913a-ae03c6ed399c admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:23.121804 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1075/2163] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:23 2022] PUT /identity/v3/projects/895ce7c59fd34d31af3bfec5f7546743/users/149fdf407b624d90b476fca9bcfcec0f/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:23.126239 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25295b17-67f9-4080-9c8b-7778c7ca7e66 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:23.126546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25295b17-67f9-4080-9c8b-7778c7ca7e66 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:23.126809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25295b17-67f9-4080-9c8b-7778c7ca7e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:23.155106 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-25295b17-67f9-4080-9c8b-7778c7ca7e66 None None] MFA Rules not processed for user `149fdf407b624d90b476fca9bcfcec0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:23.156355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-25295b17-67f9-4080-9c8b-7778c7ca7e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:23.231870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1089/2164] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:23.246577 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450c496-6f19-4c54-913a-ae03c6ed399c admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:23.326996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450c496-6f19-4c54-913a-ae03c6ed399c admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:23.331461 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8840d56-1bb7-4bde-8748-79eec41a0991 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:23.332248 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8840d56-1bb7-4bde-8748-79eec41a0991 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:23.332759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8840d56-1bb7-4bde-8748-79eec41a0991 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:23.336397 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8840d56-1bb7-4bde-8748-79eec41a0991 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:23.344621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8840d56-1bb7-4bde-8748-79eec41a0991 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:23.346253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8840d56-1bb7-4bde-8748-79eec41a0991 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:23.357691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1076/2165] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:24.329922 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25295b17-67f9-4080-9c8b-7778c7ca7e66 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:24.340073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25295b17-67f9-4080-9c8b-7778c7ca7e66 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:24.345866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c2dee-4623-414a-976a-c83b1cf74498 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '149fdf407b624d90b476fca9bcfcec0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895ce7c59fd34d31af3bfec5f7546743', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:24.346528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c2dee-4623-414a-976a-c83b1cf74498 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:24.346794 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c2dee-4623-414a-976a-c83b1cf74498 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:24.347117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c2dee-4623-414a-976a-c83b1cf74498 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:24.347779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c2dee-4623-414a-976a-c83b1cf74498 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:27:24.348090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c2dee-4623-414a-976a-c83b1cf74498 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:24.350123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c2dee-4623-414a-976a-c83b1cf74498 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:24.363106 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1090/2166] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:27:24.368034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8840d56-1bb7-4bde-8748-79eec41a0991 service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:24.377090 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8840d56-1bb7-4bde-8748-79eec41a0991 service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:24.382332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f66f8-9a2d-4d48-b3c2-b86b627affa2 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '149fdf407b624d90b476fca9bcfcec0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895ce7c59fd34d31af3bfec5f7546743', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:24.382998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f66f8-9a2d-4d48-b3c2-b86b627affa2 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:24.383286 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f66f8-9a2d-4d48-b3c2-b86b627affa2 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:24.383559 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f66f8-9a2d-4d48-b3c2-b86b627affa2 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:24.384212 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f66f8-9a2d-4d48-b3c2-b86b627affa2 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:24.386136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f66f8-9a2d-4d48-b3c2-b86b627affa2 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:24.462376 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1077/2167] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:24 2022] POST /identity/v3/projects => generated 354 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:27:24.466535 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c2dee-4623-414a-976a-c83b1cf74498 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:24.480419 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c2dee-4623-414a-976a-c83b1cf74498 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:24.541870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db59f8-4ecc-4993-99f7-62c763100434 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '149fdf407b624d90b476fca9bcfcec0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895ce7c59fd34d31af3bfec5f7546743', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:24.542476 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db59f8-4ecc-4993-99f7-62c763100434 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:24.542680 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db59f8-4ecc-4993-99f7-62c763100434 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:24.542986 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db59f8-4ecc-4993-99f7-62c763100434 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:24.543673 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db59f8-4ecc-4993-99f7-62c763100434 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:27:24.543983 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db59f8-4ecc-4993-99f7-62c763100434 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:24.545592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db59f8-4ecc-4993-99f7-62c763100434 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:24.562836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1091/2168] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:27:24.602211 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f66f8-9a2d-4d48-b3c2-b86b627affa2 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:24.612612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f66f8-9a2d-4d48-b3c2-b86b627affa2 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:24.617380 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b9fa1-850a-4130-bcc9-9e03754fce41 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '149fdf407b624d90b476fca9bcfcec0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895ce7c59fd34d31af3bfec5f7546743', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:24.618122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b9fa1-850a-4130-bcc9-9e03754fce41 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:24.618479 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b9fa1-850a-4130-bcc9-9e03754fce41 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:24.618828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b9fa1-850a-4130-bcc9-9e03754fce41 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] PATH_INFO: `/v3/projects/278831b9a5be4752949e15022e124f59` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:24.620191 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b9fa1-850a-4130-bcc9-9e03754fce41 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] RBAC: Authorizing `identity:delete_project(project_id=278831b9a5be4752949e15022e124f59)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:24.622752 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b9fa1-850a-4130-bcc9-9e03754fce41 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:24.624294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:24.765170 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-259b9fa1-850a-4130-bcc9-9e03754fce41 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '278831b9a5be4752949e15022e124f59', 'request_id': 'req-259b9fa1-850a-4130-bcc9-9e03754fce41'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:24.772705 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-259b9fa1-850a-4130-bcc9-9e03754fce41 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '278831b9a5be4752949e15022e124f59', 'request_id': 'req-259b9fa1-850a-4130-bcc9-9e03754fce41'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:24.781793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-259b9fa1-850a-4130-bcc9-9e03754fce41 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '278831b9a5be4752949e15022e124f59', 'request_id': 'req-259b9fa1-850a-4130-bcc9-9e03754fce41'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:24.872276 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-259b9fa1-850a-4130-bcc9-9e03754fce41 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] The token cache is being invalidate because project 278831b9a5be4752949e15022e124f59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:24.872554 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-259b9fa1-850a-4130-bcc9-9e03754fce41 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:24.873677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-259b9fa1-850a-4130-bcc9-9e03754fce41 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:24.874706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1078/2169] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:24 2022] DELETE /identity/v3/projects/278831b9a5be4752949e15022e124f59 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:25.995182 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db59f8-4ecc-4993-99f7-62c763100434 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:25.997069 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-67db59f8-4ecc-4993-99f7-62c763100434 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:26.002568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:26.065089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db59f8-4ecc-4993-99f7-62c763100434 tempest-VolumeQuotaClassesTest-1343298732 tempest-VolumeQuotaClassesTest-1343298732-project-admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:26.123396 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962b40d-3e72-4752-88e8-d14a6da4ffc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:26.124038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962b40d-3e72-4752-88e8-d14a6da4ffc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:26.124417 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962b40d-3e72-4752-88e8-d14a6da4ffc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:26.124742 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962b40d-3e72-4752-88e8-d14a6da4ffc6 admin admin] PATH_INFO: `/v3/users/a36ed4f9eb644f73874080e951734b41` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:26.125922 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3962b40d-3e72-4752-88e8-d14a6da4ffc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a36ed4f9eb644f73874080e951734b41)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:26.127989 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3962b40d-3e72-4752-88e8-d14a6da4ffc6 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:26.376549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3962b40d-3e72-4752-88e8-d14a6da4ffc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a36ed4f9eb644f73874080e951734b41'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:26.390374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3962b40d-3e72-4752-88e8-d14a6da4ffc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a36ed4f9eb644f73874080e951734b41'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:26.476997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3962b40d-3e72-4752-88e8-d14a6da4ffc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a36ed4f9eb644f73874080e951734b41'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:26.486686 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3962b40d-3e72-4752-88e8-d14a6da4ffc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a36ed4f9eb644f73874080e951734b41'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:26.487690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3962b40d-3e72-4752-88e8-d14a6da4ffc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a36ed4f9eb644f73874080e951734b41'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:26.498075 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1092/2170] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:25 2022] DELETE /identity/v3/users/a36ed4f9eb644f73874080e951734b41 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:26.499919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28a271-9fc1-4d22-b09e-569f4b0266fa None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:26.500569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28a271-9fc1-4d22-b09e-569f4b0266fa None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:26.500569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28a271-9fc1-4d22-b09e-569f4b0266fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:26.521934 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-9e28a271-9fc1-4d22-b09e-569f4b0266fa None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:26.523888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-9e28a271-9fc1-4d22-b09e-569f4b0266fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:26.594036 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1079/2171] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:26.607235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962b40d-3e72-4752-88e8-d14a6da4ffc6 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:26.608896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3962b40d-3e72-4752-88e8-d14a6da4ffc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:26.643761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:26.668665 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962b40d-3e72-4752-88e8-d14a6da4ffc6 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:26.672217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c5e58-f556-4fef-b198-2a7069dd473d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:26.672819 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c5e58-f556-4fef-b198-2a7069dd473d service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:26.673405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c5e58-f556-4fef-b198-2a7069dd473d service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:26.673683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c5e58-f556-4fef-b198-2a7069dd473d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:26.682636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c5e58-f556-4fef-b198-2a7069dd473d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:26.684151 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c5e58-f556-4fef-b198-2a7069dd473d service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:26.693352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1093/2172] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:27.152719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28a271-9fc1-4d22-b09e-569f4b0266fa None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:27.154181 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-9e28a271-9fc1-4d22-b09e-569f4b0266fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:27.166708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28a271-9fc1-4d22-b09e-569f4b0266fa None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:27.172303 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:27.172958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:27.173242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:27.173522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6 admin admin] PATH_INFO: `/v3/projects/d63f4d20e3a74f189d2d729a072152f3` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:27.174651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d63f4d20e3a74f189d2d729a072152f3)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:27.176727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:27.281518 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd63f4d20e3a74f189d2d729a072152f3', 'request_id': 'req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.287207 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd63f4d20e3a74f189d2d729a072152f3', 'request_id': 'req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.294389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd63f4d20e3a74f189d2d729a072152f3', 'request_id': 'req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.382967 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6 admin admin] The token cache is being invalidate because project d63f4d20e3a74f189d2d729a072152f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:27.383268 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.383915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.385429 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1080/2173] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:27 2022] DELETE /identity/v3/projects/d63f4d20e3a74f189d2d729a072152f3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:27.389467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c5e58-f556-4fef-b198-2a7069dd473d service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:27.399325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c5e58-f556-4fef-b198-2a7069dd473d service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:27.455399 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06c371-29f2-46e7-97b0-3e25b17a7b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:27.455955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06c371-29f2-46e7-97b0-3e25b17a7b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:27.456238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06c371-29f2-46e7-97b0-3e25b17a7b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:27.456515 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06c371-29f2-46e7-97b0-3e25b17a7b12 admin admin] PATH_INFO: `/v3/users/149fdf407b624d90b476fca9bcfcec0f` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:27.457653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06c371-29f2-46e7-97b0-3e25b17a7b12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=149fdf407b624d90b476fca9bcfcec0f)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:27.459621 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06c371-29f2-46e7-97b0-3e25b17a7b12 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:27.633687 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1d06c371-29f2-46e7-97b0-3e25b17a7b12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '149fdf407b624d90b476fca9bcfcec0f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.646073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1d06c371-29f2-46e7-97b0-3e25b17a7b12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '149fdf407b624d90b476fca9bcfcec0f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.711074 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1d06c371-29f2-46e7-97b0-3e25b17a7b12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '149fdf407b624d90b476fca9bcfcec0f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.718639 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1d06c371-29f2-46e7-97b0-3e25b17a7b12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '149fdf407b624d90b476fca9bcfcec0f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.719303 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1d06c371-29f2-46e7-97b0-3e25b17a7b12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '149fdf407b624d90b476fca9bcfcec0f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.727092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1094/2174] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:27 2022] DELETE /identity/v3/users/149fdf407b624d90b476fca9bcfcec0f => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:28.180902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:28.182476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:28.198481 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5e6f6-a990-44d8-8995-f4aeea1ba6a6 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:28.208861 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:28.258348 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36d0e3-17a5-43ec-9ff5-47cddf266f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:28.258920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36d0e3-17a5-43ec-9ff5-47cddf266f49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:28.259207 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36d0e3-17a5-43ec-9ff5-47cddf266f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:28.259479 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36d0e3-17a5-43ec-9ff5-47cddf266f49 admin admin] PATH_INFO: `/v3/projects/895ce7c59fd34d31af3bfec5f7546743` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:28.260618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36d0e3-17a5-43ec-9ff5-47cddf266f49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=895ce7c59fd34d31af3bfec5f7546743)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:28.262523 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36d0e3-17a5-43ec-9ff5-47cddf266f49 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:28.369872 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8a36d0e3-17a5-43ec-9ff5-47cddf266f49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '895ce7c59fd34d31af3bfec5f7546743', 'request_id': 'req-8a36d0e3-17a5-43ec-9ff5-47cddf266f49'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:28.376181 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8a36d0e3-17a5-43ec-9ff5-47cddf266f49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '895ce7c59fd34d31af3bfec5f7546743', 'request_id': 'req-8a36d0e3-17a5-43ec-9ff5-47cddf266f49'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:28.382447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8a36d0e3-17a5-43ec-9ff5-47cddf266f49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '895ce7c59fd34d31af3bfec5f7546743', 'request_id': 'req-8a36d0e3-17a5-43ec-9ff5-47cddf266f49'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:28.468623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8a36d0e3-17a5-43ec-9ff5-47cddf266f49 admin admin] The token cache is being invalidate because project 895ce7c59fd34d31af3bfec5f7546743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:28.468922 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8a36d0e3-17a5-43ec-9ff5-47cddf266f49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:28.469587 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8a36d0e3-17a5-43ec-9ff5-47cddf266f49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:28.484022 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1081/2175] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:28 2022] DELETE /identity/v3/projects/895ce7c59fd34d31af3bfec5f7546743 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:28.484147 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622a472-89a0-408d-a5af-fa1eb7f69d07 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:28.484147 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622a472-89a0-408d-a5af-fa1eb7f69d07 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:28.484147 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622a472-89a0-408d-a5af-fa1eb7f69d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:28.497423 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-8622a472-89a0-408d-a5af-fa1eb7f69d07 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:28.498616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8622a472-89a0-408d-a5af-fa1eb7f69d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:28.545543 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:28.559865 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1095/2176] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:27:28.567779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07704c84-04e6-44eb-b319-d0199f22980a None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:28.568069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07704c84-04e6-44eb-b319-d0199f22980a None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:28.568349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07704c84-04e6-44eb-b319-d0199f22980a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:28.589800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-07704c84-04e6-44eb-b319-d0199f22980a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:28.591091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-07704c84-04e6-44eb-b319-d0199f22980a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:28.605718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1082/2177] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:28.607068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622a472-89a0-408d-a5af-fa1eb7f69d07 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:28.617125 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622a472-89a0-408d-a5af-fa1eb7f69d07 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:28.621718 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba114647-a769-4221-8ceb-a8948a2c18ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:28.622314 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba114647-a769-4221-8ceb-a8948a2c18ad admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:28.622595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba114647-a769-4221-8ceb-a8948a2c18ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:28.622884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba114647-a769-4221-8ceb-a8948a2c18ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:28.623575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba114647-a769-4221-8ceb-a8948a2c18ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:27:28.623877 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba114647-a769-4221-8ceb-a8948a2c18ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:28.626844 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba114647-a769-4221-8ceb-a8948a2c18ad admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:28.637958 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1096/2178] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:27:28.642328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07704c84-04e6-44eb-b319-d0199f22980a None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:28.652648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07704c84-04e6-44eb-b319-d0199f22980a None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:28.657780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110e389-0643-49e8-a7db-dd81f3421131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:28.658396 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110e389-0643-49e8-a7db-dd81f3421131 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:28.658709 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110e389-0643-49e8-a7db-dd81f3421131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:28.659093 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110e389-0643-49e8-a7db-dd81f3421131 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:28.659744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110e389-0643-49e8-a7db-dd81f3421131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:28.661629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110e389-0643-49e8-a7db-dd81f3421131 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:28.756588 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1083/2179] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:28 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:28.761159 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba114647-a769-4221-8ceb-a8948a2c18ad admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:28.771353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba114647-a769-4221-8ceb-a8948a2c18ad admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:28.827439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab77e9-ec07-4cd3-a366-116a316c5b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:28.828048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab77e9-ec07-4cd3-a366-116a316c5b9f admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:28.828323 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab77e9-ec07-4cd3-a366-116a316c5b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:28.828598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab77e9-ec07-4cd3-a366-116a316c5b9f admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:28.829234 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab77e9-ec07-4cd3-a366-116a316c5b9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:28.831188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab77e9-ec07-4cd3-a366-116a316c5b9f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:28.954398 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1097/2180] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:28 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:28.955731 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110e389-0643-49e8-a7db-dd81f3421131 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:28.967423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110e389-0643-49e8-a7db-dd81f3421131 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:28.972142 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718718b0-1502-43f2-94a2-a8f6940a3c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:28.972895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718718b0-1502-43f2-94a2-a8f6940a3c49 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:28.973051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718718b0-1502-43f2-94a2-a8f6940a3c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:28.973323 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718718b0-1502-43f2-94a2-a8f6940a3c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:28.973970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718718b0-1502-43f2-94a2-a8f6940a3c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:28.975847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718718b0-1502-43f2-94a2-a8f6940a3c49 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:28.986431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1084/2181] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:28.990853 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab77e9-ec07-4cd3-a366-116a316c5b9f admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:29.001222 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab77e9-ec07-4cd3-a366-116a316c5b9f admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:29.005942 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997be3e-6809-4613-8ff8-244aadd99d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:29.006538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997be3e-6809-4613-8ff8-244aadd99d23 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:29.006816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997be3e-6809-4613-8ff8-244aadd99d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:29.007120 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997be3e-6809-4613-8ff8-244aadd99d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:29.008313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997be3e-6809-4613-8ff8-244aadd99d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:29.009919 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997be3e-6809-4613-8ff8-244aadd99d23 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:29.020415 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1098/2182] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:27:29.026976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718718b0-1502-43f2-94a2-a8f6940a3c49 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:29.037108 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718718b0-1502-43f2-94a2-a8f6940a3c49 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:29.042163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd61fc2-2452-4135-91ab-d48092f92ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:29.042818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd61fc2-2452-4135-91ab-d48092f92ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:29.043209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd61fc2-2452-4135-91ab-d48092f92ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:29.043487 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd61fc2-2452-4135-91ab-d48092f92ac5 admin admin] PATH_INFO: `/v3/projects/5507dc1b4edf4909ac30d8f47c1d3a5f/users/b87c7bb673084028a64c5a78953f2b00/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:29.058561 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd61fc2-2452-4135-91ab-d48092f92ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5507dc1b4edf4909ac30d8f47c1d3a5f, user_id=b87c7bb673084028a64c5a78953f2b00, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:29.061368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd61fc2-2452-4135-91ab-d48092f92ac5 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:29.203224 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1085/2183] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:29 2022] PUT /identity/v3/projects/5507dc1b4edf4909ac30d8f47c1d3a5f/users/b87c7bb673084028a64c5a78953f2b00/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:27:29.214562 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c563c-ab07-4499-ac08-1f983442bfde None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:29.217462 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c563c-ab07-4499-ac08-1f983442bfde None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:29.217462 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c563c-ab07-4499-ac08-1f983442bfde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:29.244092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-701c563c-ab07-4499-ac08-1f983442bfde None None] MFA Rules not processed for user `b87c7bb673084028a64c5a78953f2b00`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:29.245307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-701c563c-ab07-4499-ac08-1f983442bfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:29.311143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1099/2184] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:29.315598 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd61fc2-2452-4135-91ab-d48092f92ac5 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:29.358265 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:29.377725 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd61fc2-2452-4135-91ab-d48092f92ac5 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:29.382956 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdabb020-6ec0-48ae-9a85-314218ff4f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:29.383563 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabb020-6ec0-48ae-9a85-314218ff4f06 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:29.383843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabb020-6ec0-48ae-9a85-314218ff4f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:29.384111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabb020-6ec0-48ae-9a85-314218ff4f06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:29.384747 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdabb020-6ec0-48ae-9a85-314218ff4f06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:29.386621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdabb020-6ec0-48ae-9a85-314218ff4f06 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:29.472082 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1086/2185] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:29 2022] POST /identity/v3/projects => generated 387 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:29.477429 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c563c-ab07-4499-ac08-1f983442bfde None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:29.488403 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c563c-ab07-4499-ac08-1f983442bfde None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:29.532870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:29.546699 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea1493-dd21-449e-919c-8aa03a2b517d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:29.547340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea1493-dd21-449e-919c-8aa03a2b517d admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:29.547633 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea1493-dd21-449e-919c-8aa03a2b517d admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:29.547906 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea1493-dd21-449e-919c-8aa03a2b517d admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:29.548538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea1493-dd21-449e-919c-8aa03a2b517d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:29.550560 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea1493-dd21-449e-919c-8aa03a2b517d admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:29.642198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1100/2186] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:29 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:29.646302 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdabb020-6ec0-48ae-9a85-314218ff4f06 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:29.656940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdabb020-6ec0-48ae-9a85-314218ff4f06 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:29.661588 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2179d-ea23-4564-a128-027b411228e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:29.662195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2179d-ea23-4564-a128-027b411228e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:29.662506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2179d-ea23-4564-a128-027b411228e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:29.662783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2179d-ea23-4564-a128-027b411228e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:29.663973 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2179d-ea23-4564-a128-027b411228e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:29.665633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2179d-ea23-4564-a128-027b411228e8 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:29.676882 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1087/2187] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:29.681471 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea1493-dd21-449e-919c-8aa03a2b517d admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:29.691788 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea1493-dd21-449e-919c-8aa03a2b517d admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:29.696789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad742715-79b1-411e-956a-e5619aa3144d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:29.697539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad742715-79b1-411e-956a-e5619aa3144d admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:29.697904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad742715-79b1-411e-956a-e5619aa3144d admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:29.698240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad742715-79b1-411e-956a-e5619aa3144d admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:29.699050 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad742715-79b1-411e-956a-e5619aa3144d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:29.700779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad742715-79b1-411e-956a-e5619aa3144d admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:29.711645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1101/2188] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:29.715717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2179d-ea23-4564-a128-027b411228e8 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:29.725895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2179d-ea23-4564-a128-027b411228e8 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:29.731845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0be63-0032-40a0-a9d3-4710d6a0e9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:29.732502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0be63-0032-40a0-a9d3-4710d6a0e9a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:29.732779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0be63-0032-40a0-a9d3-4710d6a0e9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:29.733367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0be63-0032-40a0-a9d3-4710d6a0e9a6 admin admin] PATH_INFO: `/v3/projects/66994f4339c545b8b5a6c2ba062ac0f3/users/6fab8985055741baa1fcb3409c8d20c9/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:29.748476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0be63-0032-40a0-a9d3-4710d6a0e9a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66994f4339c545b8b5a6c2ba062ac0f3, user_id=6fab8985055741baa1fcb3409c8d20c9, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:29.751193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0be63-0032-40a0-a9d3-4710d6a0e9a6 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:29.819739 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1088/2189] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:29 2022] PUT /identity/v3/projects/66994f4339c545b8b5a6c2ba062ac0f3/users/6fab8985055741baa1fcb3409c8d20c9/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:29.831773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069751b2-a651-44aa-a33f-5d207a82e83a None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:29.832065 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069751b2-a651-44aa-a33f-5d207a82e83a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:29.832340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069751b2-a651-44aa-a33f-5d207a82e83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:29.861333 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-069751b2-a651-44aa-a33f-5d207a82e83a None None] MFA Rules not processed for user `6fab8985055741baa1fcb3409c8d20c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:29.862550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-069751b2-a651-44aa-a33f-5d207a82e83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:29.926809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1102/2190] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:29.931493 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0be63-0032-40a0-a9d3-4710d6a0e9a6 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:29.994130 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0be63-0032-40a0-a9d3-4710d6a0e9a6 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:29.998845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bed24-ac9d-4044-8ee4-a7c648b4268b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:29.999486 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bed24-ac9d-4044-8ee4-a7c648b4268b admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:29.999771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bed24-ac9d-4044-8ee4-a7c648b4268b admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:30.000045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bed24-ac9d-4044-8ee4-a7c648b4268b admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:30.000687 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18bed24-ac9d-4044-8ee4-a7c648b4268b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:30.002588 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18bed24-ac9d-4044-8ee4-a7c648b4268b admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:30.135608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1089/2191] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:29 2022] POST /identity/v3/projects => generated 385 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:30.139747 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069751b2-a651-44aa-a33f-5d207a82e83a None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:30.151741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069751b2-a651-44aa-a33f-5d207a82e83a None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:30.209774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94388f-8e64-4bef-a2fa-5bf467f84bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:30.211405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94388f-8e64-4bef-a2fa-5bf467f84bfe admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:30.211558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94388f-8e64-4bef-a2fa-5bf467f84bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:30.211960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94388f-8e64-4bef-a2fa-5bf467f84bfe admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:30.212541 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94388f-8e64-4bef-a2fa-5bf467f84bfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:30.214471 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94388f-8e64-4bef-a2fa-5bf467f84bfe admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:30.315018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1103/2192] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:30 2022] POST /identity/v3/users => generated 299 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:30.319246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bed24-ac9d-4044-8ee4-a7c648b4268b admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:30.331271 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bed24-ac9d-4044-8ee4-a7c648b4268b admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:30.336926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384c38f-aede-4a83-82c5-a933a99aa29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:30.337544 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384c38f-aede-4a83-82c5-a933a99aa29f admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:30.337826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384c38f-aede-4a83-82c5-a933a99aa29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:30.338106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384c38f-aede-4a83-82c5-a933a99aa29f admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:30.338743 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384c38f-aede-4a83-82c5-a933a99aa29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:30.340378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384c38f-aede-4a83-82c5-a933a99aa29f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:30.352078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1090/2193] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:30.356087 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94388f-8e64-4bef-a2fa-5bf467f84bfe admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:30.366193 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94388f-8e64-4bef-a2fa-5bf467f84bfe admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:30.371136 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cd65f-8c59-46ef-9900-6bff6d5b862a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:30.371784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cd65f-8c59-46ef-9900-6bff6d5b862a admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:30.372067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cd65f-8c59-46ef-9900-6bff6d5b862a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:30.372345 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cd65f-8c59-46ef-9900-6bff6d5b862a admin admin] PATH_INFO: `/v3/projects/659e0bd1050a4d428b7e69550cf6ae05/users/e9caf6dc123d424f8d633ef284867f81/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:30.388848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cd65f-8c59-46ef-9900-6bff6d5b862a admin admin] RBAC: Authorizing `identity:create_grant(project_id=659e0bd1050a4d428b7e69550cf6ae05, user_id=e9caf6dc123d424f8d633ef284867f81, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:30.391554 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cd65f-8c59-46ef-9900-6bff6d5b862a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:30.482484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1104/2194] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:30 2022] PUT /identity/v3/projects/659e0bd1050a4d428b7e69550cf6ae05/users/e9caf6dc123d424f8d633ef284867f81/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:30.486405 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384c38f-aede-4a83-82c5-a933a99aa29f admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:30.496692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384c38f-aede-4a83-82c5-a933a99aa29f admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:30.553800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a45301-5670-48b7-bf79-7ea7237c1195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:30.554508 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a45301-5670-48b7-bf79-7ea7237c1195 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:30.554792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a45301-5670-48b7-bf79-7ea7237c1195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:30.555111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a45301-5670-48b7-bf79-7ea7237c1195 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:30.555751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a45301-5670-48b7-bf79-7ea7237c1195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:30.557371 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a45301-5670-48b7-bf79-7ea7237c1195 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:30.562397 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:30.567430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1091/2195] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:30.572241 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cd65f-8c59-46ef-9900-6bff6d5b862a admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:30.582577 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cd65f-8c59-46ef-9900-6bff6d5b862a admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:30.587993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04988699-c1ea-420c-a44c-a9a67f102c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:30.588639 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04988699-c1ea-420c-a44c-a9a67f102c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:30.588917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04988699-c1ea-420c-a44c-a9a67f102c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:30.589187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04988699-c1ea-420c-a44c-a9a67f102c2c admin admin] PATH_INFO: `/v3/projects/659e0bd1050a4d428b7e69550cf6ae05/users/e9caf6dc123d424f8d633ef284867f81/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:30.591132 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04988699-c1ea-420c-a44c-a9a67f102c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=659e0bd1050a4d428b7e69550cf6ae05, user_id=e9caf6dc123d424f8d633ef284867f81, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:30.593739 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04988699-c1ea-420c-a44c-a9a67f102c2c admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:30.605564 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1105/2196] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:30 2022] PUT /identity/v3/projects/659e0bd1050a4d428b7e69550cf6ae05/users/e9caf6dc123d424f8d633ef284867f81/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:27:30.617624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f15f4a-106b-481e-8912-f7593c1be36e None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:30.617913 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f15f4a-106b-481e-8912-f7593c1be36e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:30.618194 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f15f4a-106b-481e-8912-f7593c1be36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:30.649834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-72f15f4a-106b-481e-8912-f7593c1be36e None None] MFA Rules not processed for user `e9caf6dc123d424f8d633ef284867f81`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:30.651126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-72f15f4a-106b-481e-8912-f7593c1be36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:30.721505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1092/2197] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:30.732984 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04988699-c1ea-420c-a44c-a9a67f102c2c admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:30.734681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-04988699-c1ea-420c-a44c-a9a67f102c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:30.745321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:30.795526 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04988699-c1ea-420c-a44c-a9a67f102c2c admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:30.799179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cad9a-d369-4d0f-89df-da19e608dd75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:30.799785 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cad9a-d369-4d0f-89df-da19e608dd75 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:30.800064 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cad9a-d369-4d0f-89df-da19e608dd75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:30.800337 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cad9a-d369-4d0f-89df-da19e608dd75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:30.811086 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508cad9a-d369-4d0f-89df-da19e608dd75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:30.812521 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508cad9a-d369-4d0f-89df-da19e608dd75 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:30.823012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1106/2198] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:31.889868 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f15f4a-106b-481e-8912-f7593c1be36e None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:31.891483 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-72f15f4a-106b-481e-8912-f7593c1be36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:31.904370 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f15f4a-106b-481e-8912-f7593c1be36e None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:31.907982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ed895-17ef-4cb1-adbb-fab7857213f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:31.908583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ed895-17ef-4cb1-adbb-fab7857213f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:31.908878 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ed895-17ef-4cb1-adbb-fab7857213f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:31.909191 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ed895-17ef-4cb1-adbb-fab7857213f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:31.967182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ed895-17ef-4cb1-adbb-fab7857213f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:31.968535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ed895-17ef-4cb1-adbb-fab7857213f5 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:31.977854 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1093/2199] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:40.818127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=23 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:41.153949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cad9a-d369-4d0f-89df-da19e608dd75 service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:41.155487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-508cad9a-d369-4d0f-89df-da19e608dd75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:41.217716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ed895-17ef-4cb1-adbb-fab7857213f5 service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:41.222201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cad9a-d369-4d0f-89df-da19e608dd75 service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:41.227426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ed895-17ef-4cb1-adbb-fab7857213f5 service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:41.228181 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b21d7e-87dc-491b-b455-2158143f823c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:41.228891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b21d7e-87dc-491b-b455-2158143f823c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:41.229171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b21d7e-87dc-491b-b455-2158143f823c admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:41.229450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b21d7e-87dc-491b-b455-2158143f823c admin admin] PATH_INFO: `/v3/users/2fb4a631005d4a00aa942d6652ba35c0` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:41.230711 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b21d7e-87dc-491b-b455-2158143f823c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fb4a631005d4a00aa942d6652ba35c0)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:41.231165 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4460a65-bb4a-4af3-a624-6b191bc2d773 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:41.231759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4460a65-bb4a-4af3-a624-6b191bc2d773 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:41.232036 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4460a65-bb4a-4af3-a624-6b191bc2d773 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:41.232305 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4460a65-bb4a-4af3-a624-6b191bc2d773 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:41.232829 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b21d7e-87dc-491b-b455-2158143f823c admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:41.291612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4460a65-bb4a-4af3-a624-6b191bc2d773 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:41.293036 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4460a65-bb4a-4af3-a624-6b191bc2d773 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:41.302155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1094/2200] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:41 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:27:41.421286 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-62b21d7e-87dc-491b-b455-2158143f823c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fb4a631005d4a00aa942d6652ba35c0'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:41.434122 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-62b21d7e-87dc-491b-b455-2158143f823c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fb4a631005d4a00aa942d6652ba35c0'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:41.533135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-62b21d7e-87dc-491b-b455-2158143f823c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fb4a631005d4a00aa942d6652ba35c0'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:41.541501 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-62b21d7e-87dc-491b-b455-2158143f823c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fb4a631005d4a00aa942d6652ba35c0'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:41.542157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-62b21d7e-87dc-491b-b455-2158143f823c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fb4a631005d4a00aa942d6652ba35c0'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:41.548084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1107/2201] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:41 2022] DELETE /identity/v3/users/2fb4a631005d4a00aa942d6652ba35c0 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:41.552799 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb643b0a-4916-4afe-bc8b-2eb69bddbdc0 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:41.553089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb643b0a-4916-4afe-bc8b-2eb69bddbdc0 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:41.553371 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb643b0a-4916-4afe-bc8b-2eb69bddbdc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:41.574019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-fb643b0a-4916-4afe-bc8b-2eb69bddbdc0 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:41.575300 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-fb643b0a-4916-4afe-bc8b-2eb69bddbdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:41.593285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:41.652042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1095/2202] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:41.659244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b21d7e-87dc-491b-b455-2158143f823c admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:41.661020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-62b21d7e-87dc-491b-b455-2158143f823c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:41.729284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b21d7e-87dc-491b-b455-2158143f823c admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:41.733497 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98e61a-83ca-41f1-b7a0-1aff275de643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:41.734148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98e61a-83ca-41f1-b7a0-1aff275de643 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:41.734438 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98e61a-83ca-41f1-b7a0-1aff275de643 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:41.734718 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98e61a-83ca-41f1-b7a0-1aff275de643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:41.745202 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98e61a-83ca-41f1-b7a0-1aff275de643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:41.747035 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98e61a-83ca-41f1-b7a0-1aff275de643 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:41.758038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1108/2203] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:27:42.227213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb643b0a-4916-4afe-bc8b-2eb69bddbdc0 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:42.228712 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-fb643b0a-4916-4afe-bc8b-2eb69bddbdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:42.241827 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb643b0a-4916-4afe-bc8b-2eb69bddbdc0 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:42.246972 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89c140-7dc0-4bfb-bcad-63f053feb8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:42.247637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89c140-7dc0-4bfb-bcad-63f053feb8ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:42.247923 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89c140-7dc0-4bfb-bcad-63f053feb8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:42.248198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89c140-7dc0-4bfb-bcad-63f053feb8ca admin admin] PATH_INFO: `/v3/projects/75e71c290eac4fe393eec1d1d8ee2a59` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:42.249386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89c140-7dc0-4bfb-bcad-63f053feb8ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=75e71c290eac4fe393eec1d1d8ee2a59)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:42.251643 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89c140-7dc0-4bfb-bcad-63f053feb8ca admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:42.373288 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6e89c140-7dc0-4bfb-bcad-63f053feb8ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75e71c290eac4fe393eec1d1d8ee2a59', 'request_id': 'req-6e89c140-7dc0-4bfb-bcad-63f053feb8ca'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:42.378668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6e89c140-7dc0-4bfb-bcad-63f053feb8ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75e71c290eac4fe393eec1d1d8ee2a59', 'request_id': 'req-6e89c140-7dc0-4bfb-bcad-63f053feb8ca'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:42.385649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6e89c140-7dc0-4bfb-bcad-63f053feb8ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75e71c290eac4fe393eec1d1d8ee2a59', 'request_id': 'req-6e89c140-7dc0-4bfb-bcad-63f053feb8ca'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:42.458147 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6e89c140-7dc0-4bfb-bcad-63f053feb8ca admin admin] The token cache is being invalidate because project 75e71c290eac4fe393eec1d1d8ee2a59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:42.458433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6e89c140-7dc0-4bfb-bcad-63f053feb8ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:42.459081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6e89c140-7dc0-4bfb-bcad-63f053feb8ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:42.461314 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1096/2204] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:42 2022] DELETE /identity/v3/projects/75e71c290eac4fe393eec1d1d8ee2a59 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:42.466266 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c0c81-a1a6-4405-bee9-1c81e5da3342 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:42.467015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c0c81-a1a6-4405-bee9-1c81e5da3342 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:42.467015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c0c81-a1a6-4405-bee9-1c81e5da3342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:42.487619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-e15c0c81-a1a6-4405-bee9-1c81e5da3342 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:42.488807 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e15c0c81-a1a6-4405-bee9-1c81e5da3342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:42.507948 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:42.565828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1109/2205] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:42.574337 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1550ab-0be8-490c-a21d-eacae70cb8e7 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:42.574629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1550ab-0be8-490c-a21d-eacae70cb8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:42.574938 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1550ab-0be8-490c-a21d-eacae70cb8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:42.597262 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ac1550ab-0be8-490c-a21d-eacae70cb8e7 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:42.598518 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ac1550ab-0be8-490c-a21d-eacae70cb8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:42.607189 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1097/2206] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:42.611468 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c0c81-a1a6-4405-bee9-1c81e5da3342 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:42.622391 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c0c81-a1a6-4405-bee9-1c81e5da3342 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:42.627095 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee9dd9-d26d-4116-a3c6-6052a0de1d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:42.627634 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee9dd9-d26d-4116-a3c6-6052a0de1d3e admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:42.627914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee9dd9-d26d-4116-a3c6-6052a0de1d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:42.628188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee9dd9-d26d-4116-a3c6-6052a0de1d3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:42.628888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee9dd9-d26d-4116-a3c6-6052a0de1d3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:27:42.629193 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee9dd9-d26d-4116-a3c6-6052a0de1d3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:42.630804 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee9dd9-d26d-4116-a3c6-6052a0de1d3e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:42.643282 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1110/2207] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:27:42.647723 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1550ab-0be8-490c-a21d-eacae70cb8e7 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:42.656993 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1550ab-0be8-490c-a21d-eacae70cb8e7 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:42.661773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc00f9-9716-4414-b401-c1bf281d3a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:42.662394 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc00f9-9716-4414-b401-c1bf281d3a79 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:42.662684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc00f9-9716-4414-b401-c1bf281d3a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:42.663163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc00f9-9716-4414-b401-c1bf281d3a79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:42.663800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc00f9-9716-4414-b401-c1bf281d3a79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:42.665693 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc00f9-9716-4414-b401-c1bf281d3a79 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:42.750610 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1098/2208] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:42 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:42.754639 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee9dd9-d26d-4116-a3c6-6052a0de1d3e admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:42.765300 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee9dd9-d26d-4116-a3c6-6052a0de1d3e admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:42.823010 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafdb50-6b04-49be-9bb0-76d72266ffea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:42.823541 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafdb50-6b04-49be-9bb0-76d72266ffea admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:42.823825 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafdb50-6b04-49be-9bb0-76d72266ffea admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:42.824101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafdb50-6b04-49be-9bb0-76d72266ffea admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:42.824741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafdb50-6b04-49be-9bb0-76d72266ffea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:42.826642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafdb50-6b04-49be-9bb0-76d72266ffea admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:42.905661 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1111/2209] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:42 2022] POST /identity/v3/users => generated 287 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:42.909871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc00f9-9716-4414-b401-c1bf281d3a79 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:42.921078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc00f9-9716-4414-b401-c1bf281d3a79 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:42.925946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94e430-3c6d-4ed4-a489-033ff14bfa78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:42.926693 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94e430-3c6d-4ed4-a489-033ff14bfa78 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:42.927028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94e430-3c6d-4ed4-a489-033ff14bfa78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:42.927307 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94e430-3c6d-4ed4-a489-033ff14bfa78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:42.927988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94e430-3c6d-4ed4-a489-033ff14bfa78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:42.930465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94e430-3c6d-4ed4-a489-033ff14bfa78 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:42.941768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1099/2210] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:42.946686 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafdb50-6b04-49be-9bb0-76d72266ffea admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:42.957041 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafdb50-6b04-49be-9bb0-76d72266ffea admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:42.961899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88430952-236c-4519-8dc2-b2a0df68a5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:42.966827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88430952-236c-4519-8dc2-b2a0df68a5ce admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:42.966827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88430952-236c-4519-8dc2-b2a0df68a5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:42.966827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88430952-236c-4519-8dc2-b2a0df68a5ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:42.966827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88430952-236c-4519-8dc2-b2a0df68a5ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:42.966827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88430952-236c-4519-8dc2-b2a0df68a5ce admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:42.978631 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1112/2211] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:42.983078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94e430-3c6d-4ed4-a489-033ff14bfa78 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:42.994089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94e430-3c6d-4ed4-a489-033ff14bfa78 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:42.998844 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d50c7-f2b9-42b3-baca-0ff4a70aad2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:42.999529 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d50c7-f2b9-42b3-baca-0ff4a70aad2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:42.999808 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d50c7-f2b9-42b3-baca-0ff4a70aad2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:43.000081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d50c7-f2b9-42b3-baca-0ff4a70aad2a admin admin] PATH_INFO: `/v3/projects/7f1096f5ccd9443192c4d91d8984a633/users/1cc4dfb3be554d6e8b419a01f7f608c4/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:43.015150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d50c7-f2b9-42b3-baca-0ff4a70aad2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f1096f5ccd9443192c4d91d8984a633, user_id=1cc4dfb3be554d6e8b419a01f7f608c4, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:43.017824 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d50c7-f2b9-42b3-baca-0ff4a70aad2a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:43.122802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1100/2212] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:42 2022] PUT /identity/v3/projects/7f1096f5ccd9443192c4d91d8984a633/users/1cc4dfb3be554d6e8b419a01f7f608c4/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:43.133628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c6350-d21a-4800-a219-0043cef84c57 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:43.133913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c6350-d21a-4800-a219-0043cef84c57 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:43.134188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c6350-d21a-4800-a219-0043cef84c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:43.162471 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-260c6350-d21a-4800-a219-0043cef84c57 None None] MFA Rules not processed for user `1cc4dfb3be554d6e8b419a01f7f608c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:43.164139 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-260c6350-d21a-4800-a219-0043cef84c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:43.234537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1113/2213] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:43.245617 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d50c7-f2b9-42b3-baca-0ff4a70aad2a admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:43.247289 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-bf7d50c7-f2b9-42b3-baca-0ff4a70aad2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:43.299448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:43.312375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d50c7-f2b9-42b3-baca-0ff4a70aad2a admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:43.316297 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbed78-7e73-4e71-bb0e-ccfe0fabab82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:43.316912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbed78-7e73-4e71-bb0e-ccfe0fabab82 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:43.317190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbed78-7e73-4e71-bb0e-ccfe0fabab82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:43.317483 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbed78-7e73-4e71-bb0e-ccfe0fabab82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:43.326458 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbed78-7e73-4e71-bb0e-ccfe0fabab82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:43.327950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbed78-7e73-4e71-bb0e-ccfe0fabab82 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:43.337317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1101/2214] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:48.147514 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663da9d-3abf-40ab-9411-732b749c3439 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:48.148048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663da9d-3abf-40ab-9411-732b749c3439 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:48.148048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663da9d-3abf-40ab-9411-732b749c3439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:48.168800 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-8663da9d-3abf-40ab-9411-732b749c3439 None None] MFA Rules not processed for user `be55f0bb21f044b0a8346cf953b1650e`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:48.170062 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8663da9d-3abf-40ab-9411-732b749c3439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:48.182187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:48.226232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1114/2215] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:27:49.500341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbed78-7e73-4e71-bb0e-ccfe0fabab82 service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:49.501790 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4cfbed78-7e73-4e71-bb0e-ccfe0fabab82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:49.596542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbed78-7e73-4e71-bb0e-ccfe0fabab82 service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:49.602213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83388489-7b1c-4869-8e8d-87b374618b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:49.602533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83388489-7b1c-4869-8e8d-87b374618b4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:49.602909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83388489-7b1c-4869-8e8d-87b374618b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:49.603394 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83388489-7b1c-4869-8e8d-87b374618b4a admin admin] PATH_INFO: `/v3/users/b87c7bb673084028a64c5a78953f2b00` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:49.604671 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83388489-7b1c-4869-8e8d-87b374618b4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b87c7bb673084028a64c5a78953f2b00)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:49.606796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83388489-7b1c-4869-8e8d-87b374618b4a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:49.811679 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-83388489-7b1c-4869-8e8d-87b374618b4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b87c7bb673084028a64c5a78953f2b00'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:49.908016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-83388489-7b1c-4869-8e8d-87b374618b4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b87c7bb673084028a64c5a78953f2b00'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:49.908687 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-83388489-7b1c-4869-8e8d-87b374618b4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b87c7bb673084028a64c5a78953f2b00'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:49.926372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-83388489-7b1c-4869-8e8d-87b374618b4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b87c7bb673084028a64c5a78953f2b00'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:49.927800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-83388489-7b1c-4869-8e8d-87b374618b4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b87c7bb673084028a64c5a78953f2b00'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:49.958748 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1102/2216] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:49 2022] DELETE /identity/v3/users/b87c7bb673084028a64c5a78953f2b00 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:49.964107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea97e1e-d9db-497b-9627-97a16129cd48 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:49.964588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea97e1e-d9db-497b-9627-97a16129cd48 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:49.964684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea97e1e-d9db-497b-9627-97a16129cd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:49.986244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-5ea97e1e-d9db-497b-9627-97a16129cd48 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:49.987445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-5ea97e1e-d9db-497b-9627-97a16129cd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:50.069510 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1115/2217] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:50.080549 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83388489-7b1c-4869-8e8d-87b374618b4a admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:50.082275 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-83388489-7b1c-4869-8e8d-87b374618b4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:50.145857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83388489-7b1c-4869-8e8d-87b374618b4a admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:50.150346 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1281c3-ca8c-4042-8ae3-9c03e1f27fac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:50.150997 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1281c3-ca8c-4042-8ae3-9c03e1f27fac service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:50.151281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1281c3-ca8c-4042-8ae3-9c03e1f27fac service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:50.151561 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1281c3-ca8c-4042-8ae3-9c03e1f27fac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:50.161310 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1281c3-ca8c-4042-8ae3-9c03e1f27fac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:50.162815 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1281c3-ca8c-4042-8ae3-9c03e1f27fac service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:50.173746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1103/2218] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:50.632464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea97e1e-d9db-497b-9627-97a16129cd48 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:50.632673 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-5ea97e1e-d9db-497b-9627-97a16129cd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:50.653246 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea97e1e-d9db-497b-9627-97a16129cd48 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:50.661369 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:50.661768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:50.662064 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:50.662344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209 admin admin] PATH_INFO: `/v3/projects/5507dc1b4edf4909ac30d8f47c1d3a5f` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:50.667439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5507dc1b4edf4909ac30d8f47c1d3a5f)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:50.669427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:50.793328 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5507dc1b4edf4909ac30d8f47c1d3a5f', 'request_id': 'req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:50.798792 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5507dc1b4edf4909ac30d8f47c1d3a5f', 'request_id': 'req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:50.896509 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5507dc1b4edf4909ac30d8f47c1d3a5f', 'request_id': 'req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:50.908818 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209 admin admin] The token cache is being invalidate because project 5507dc1b4edf4909ac30d8f47c1d3a5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:50.909107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:50.909775 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:50.911289 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1116/2219] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:50 2022] DELETE /identity/v3/projects/5507dc1b4edf4909ac30d8f47c1d3a5f => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:50.916174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1281c3-ca8c-4042-8ae3-9c03e1f27fac service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:50.926110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1281c3-ca8c-4042-8ae3-9c03e1f27fac service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:50.929876 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:50.984838 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58566193-1ed3-4273-a2cb-76a6dfd90bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:50.985768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58566193-1ed3-4273-a2cb-76a6dfd90bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:50.986209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58566193-1ed3-4273-a2cb-76a6dfd90bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:50.986630 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58566193-1ed3-4273-a2cb-76a6dfd90bed admin admin] PATH_INFO: `/v3/users/6fab8985055741baa1fcb3409c8d20c9` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:50.988031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58566193-1ed3-4273-a2cb-76a6dfd90bed admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fab8985055741baa1fcb3409c8d20c9)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:50.990070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58566193-1ed3-4273-a2cb-76a6dfd90bed admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:51.237253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-58566193-1ed3-4273-a2cb-76a6dfd90bed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fab8985055741baa1fcb3409c8d20c9'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:51.371447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-58566193-1ed3-4273-a2cb-76a6dfd90bed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fab8985055741baa1fcb3409c8d20c9'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:51.372093 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-58566193-1ed3-4273-a2cb-76a6dfd90bed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fab8985055741baa1fcb3409c8d20c9'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:51.388770 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-58566193-1ed3-4273-a2cb-76a6dfd90bed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fab8985055741baa1fcb3409c8d20c9'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:51.389510 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-58566193-1ed3-4273-a2cb-76a6dfd90bed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fab8985055741baa1fcb3409c8d20c9'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:51.407157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1104/2220] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:50 2022] DELETE /identity/v3/users/6fab8985055741baa1fcb3409c8d20c9 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:51.884873 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:51.884873 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:51.896163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba76ca7-eabd-49e4-950e-c5e7d3ca6209 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:51.954631 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:51.969030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:51.970820 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:51.970820 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:51.970820 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e admin admin] PATH_INFO: `/v3/projects/66994f4339c545b8b5a6c2ba062ac0f3` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:51.971471 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e admin admin] RBAC: Authorizing `identity:delete_project(project_id=66994f4339c545b8b5a6c2ba062ac0f3)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:51.973453 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:52.101837 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66994f4339c545b8b5a6c2ba062ac0f3', 'request_id': 'req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.112585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66994f4339c545b8b5a6c2ba062ac0f3', 'request_id': 'req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.259035 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66994f4339c545b8b5a6c2ba062ac0f3', 'request_id': 'req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.271774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e admin admin] The token cache is being invalidate because project 66994f4339c545b8b5a6c2ba062ac0f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:52.272135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.272813 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.275522 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1117/2221] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:51 2022] DELETE /identity/v3/projects/66994f4339c545b8b5a6c2ba062ac0f3 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:52.279525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58566193-1ed3-4273-a2cb-76a6dfd90bed admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:52.287347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-58566193-1ed3-4273-a2cb-76a6dfd90bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:52.294719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58566193-1ed3-4273-a2cb-76a6dfd90bed admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:52.355550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095ec2d-5695-40f8-8277-8f3d3cd5d041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:52.356283 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095ec2d-5695-40f8-8277-8f3d3cd5d041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:52.356631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095ec2d-5695-40f8-8277-8f3d3cd5d041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:52.356976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095ec2d-5695-40f8-8277-8f3d3cd5d041 admin admin] PATH_INFO: `/v3/users/e9caf6dc123d424f8d633ef284867f81` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:52.358242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095ec2d-5695-40f8-8277-8f3d3cd5d041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9caf6dc123d424f8d633ef284867f81)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:52.360249 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095ec2d-5695-40f8-8277-8f3d3cd5d041 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:52.506196 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:52.630265 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-3095ec2d-5695-40f8-8277-8f3d3cd5d041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9caf6dc123d424f8d633ef284867f81'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.744880 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-3095ec2d-5695-40f8-8277-8f3d3cd5d041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9caf6dc123d424f8d633ef284867f81'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.745505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-3095ec2d-5695-40f8-8277-8f3d3cd5d041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9caf6dc123d424f8d633ef284867f81'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.760746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-3095ec2d-5695-40f8-8277-8f3d3cd5d041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9caf6dc123d424f8d633ef284867f81'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.761469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-3095ec2d-5695-40f8-8277-8f3d3cd5d041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9caf6dc123d424f8d633ef284867f81'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.776009 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1105/2222] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:52 2022] DELETE /identity/v3/users/e9caf6dc123d424f8d633ef284867f81 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:27:53.538630 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:53.540775 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:53.562148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88e8ac-b0d8-4e8e-be3a-84cc18b5031e admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:53.649407 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94f4ab-07dc-4638-ad91-28a25856a7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:53.650359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94f4ab-07dc-4638-ad91-28a25856a7fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:53.650938 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94f4ab-07dc-4638-ad91-28a25856a7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:53.651367 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94f4ab-07dc-4638-ad91-28a25856a7fc admin admin] PATH_INFO: `/v3/projects/659e0bd1050a4d428b7e69550cf6ae05` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:53.652911 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94f4ab-07dc-4638-ad91-28a25856a7fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=659e0bd1050a4d428b7e69550cf6ae05)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:53.654900 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94f4ab-07dc-4638-ad91-28a25856a7fc admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:53.776242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ac94f4ab-07dc-4638-ad91-28a25856a7fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '659e0bd1050a4d428b7e69550cf6ae05', 'request_id': 'req-ac94f4ab-07dc-4638-ad91-28a25856a7fc'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:53.782186 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ac94f4ab-07dc-4638-ad91-28a25856a7fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '659e0bd1050a4d428b7e69550cf6ae05', 'request_id': 'req-ac94f4ab-07dc-4638-ad91-28a25856a7fc'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:53.913635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ac94f4ab-07dc-4638-ad91-28a25856a7fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '659e0bd1050a4d428b7e69550cf6ae05', 'request_id': 'req-ac94f4ab-07dc-4638-ad91-28a25856a7fc'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:53.926579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ac94f4ab-07dc-4638-ad91-28a25856a7fc admin admin] The token cache is being invalidate because project 659e0bd1050a4d428b7e69550cf6ae05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:53.926894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ac94f4ab-07dc-4638-ad91-28a25856a7fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:53.927532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ac94f4ab-07dc-4638-ad91-28a25856a7fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:53.929031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1118/2223] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:53 2022] DELETE /identity/v3/projects/659e0bd1050a4d428b7e69550cf6ae05 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:53.935165 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefeee8-75d4-45df-aa16-cd86e51a26c8 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:53.935647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefeee8-75d4-45df-aa16-cd86e51a26c8 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:53.935739 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefeee8-75d4-45df-aa16-cd86e51a26c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:53.956792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-eaefeee8-75d4-45df-aa16-cd86e51a26c8 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:53.958009 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-eaefeee8-75d4-45df-aa16-cd86e51a26c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:54.045861 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1106/2224] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:54.055040 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5156b37-1a3f-47f0-9f90-9b6dc3f3aed6 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:54.060558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5156b37-1a3f-47f0-9f90-9b6dc3f3aed6 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:54.060558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5156b37-1a3f-47f0-9f90-9b6dc3f3aed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:54.077273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:54.084002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-b5156b37-1a3f-47f0-9f90-9b6dc3f3aed6 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:54.085595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-b5156b37-1a3f-47f0-9f90-9b6dc3f3aed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:54.096941 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1119/2225] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:54.101306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefeee8-75d4-45df-aa16-cd86e51a26c8 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:54.113435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefeee8-75d4-45df-aa16-cd86e51a26c8 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:54.123956 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d643bd-7696-4a97-a21f-07d209387380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:54.123956 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d643bd-7696-4a97-a21f-07d209387380 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:54.123956 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d643bd-7696-4a97-a21f-07d209387380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:54.123956 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d643bd-7696-4a97-a21f-07d209387380 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:54.124317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d643bd-7696-4a97-a21f-07d209387380 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:27:54.124317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d643bd-7696-4a97-a21f-07d209387380 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:54.124317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d643bd-7696-4a97-a21f-07d209387380 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:54.145120 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1107/2226] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:27:54.146562 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5156b37-1a3f-47f0-9f90-9b6dc3f3aed6 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:54.160456 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5156b37-1a3f-47f0-9f90-9b6dc3f3aed6 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:54.166013 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdeb54-ddbb-4b78-8ca1-d91fc04d16a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:54.167481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdeb54-ddbb-4b78-8ca1-d91fc04d16a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:54.168012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdeb54-ddbb-4b78-8ca1-d91fc04d16a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:54.168502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdeb54-ddbb-4b78-8ca1-d91fc04d16a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:54.169606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcdeb54-ddbb-4b78-8ca1-d91fc04d16a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:54.171871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcdeb54-ddbb-4b78-8ca1-d91fc04d16a2 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:54.359112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1120/2227] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:54 2022] POST /identity/v3/projects => generated 381 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:27:54.360375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d643bd-7696-4a97-a21f-07d209387380 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:54.371432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d643bd-7696-4a97-a21f-07d209387380 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:54.444443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a81eb-6349-48a9-a16b-8c9c6c587a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:54.445301 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a81eb-6349-48a9-a16b-8c9c6c587a8f admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:54.445686 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a81eb-6349-48a9-a16b-8c9c6c587a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:54.446671 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a81eb-6349-48a9-a16b-8c9c6c587a8f admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:54.447841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a81eb-6349-48a9-a16b-8c9c6c587a8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:54.449982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a81eb-6349-48a9-a16b-8c9c6c587a8f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:54.569398 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1108/2228] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:54 2022] POST /identity/v3/users => generated 291 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:54.575445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdeb54-ddbb-4b78-8ca1-d91fc04d16a2 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:54.588738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdeb54-ddbb-4b78-8ca1-d91fc04d16a2 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:54.597649 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58b90a-b8cc-45fe-be54-d9d96b3a93f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:54.598351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58b90a-b8cc-45fe-be54-d9d96b3a93f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:54.598914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58b90a-b8cc-45fe-be54-d9d96b3a93f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:54.599195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58b90a-b8cc-45fe-be54-d9d96b3a93f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:54.600005 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58b90a-b8cc-45fe-be54-d9d96b3a93f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:54.601936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58b90a-b8cc-45fe-be54-d9d96b3a93f3 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:54.618653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1121/2229] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:54.619976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a81eb-6349-48a9-a16b-8c9c6c587a8f admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:54.635247 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a81eb-6349-48a9-a16b-8c9c6c587a8f admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:54.640411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ac4bf-54c4-4524-91b6-b6dc8d07570d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:54.641107 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ac4bf-54c4-4524-91b6-b6dc8d07570d admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:54.641572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ac4bf-54c4-4524-91b6-b6dc8d07570d admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:54.641919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ac4bf-54c4-4524-91b6-b6dc8d07570d admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:54.642634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18ac4bf-54c4-4524-91b6-b6dc8d07570d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:54.644432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18ac4bf-54c4-4524-91b6-b6dc8d07570d admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:54.662637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1109/2230] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:27:54.667289 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58b90a-b8cc-45fe-be54-d9d96b3a93f3 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:54.677949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58b90a-b8cc-45fe-be54-d9d96b3a93f3 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:54.684227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3d4e3-f925-4f42-b810-795865c3e12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:54.684893 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3d4e3-f925-4f42-b810-795865c3e12c admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:54.685177 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3d4e3-f925-4f42-b810-795865c3e12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:54.685455 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3d4e3-f925-4f42-b810-795865c3e12c admin admin] PATH_INFO: `/v3/projects/0560bd88f0fe475cb6bc8980323943f7/users/198fc58f13c84db28ec8c69b1f184c83/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:54.701991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3d4e3-f925-4f42-b810-795865c3e12c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0560bd88f0fe475cb6bc8980323943f7, user_id=198fc58f13c84db28ec8c69b1f184c83, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:54.704769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3d4e3-f925-4f42-b810-795865c3e12c admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:54.779597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1122/2231] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:54 2022] PUT /identity/v3/projects/0560bd88f0fe475cb6bc8980323943f7/users/198fc58f13c84db28ec8c69b1f184c83/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:54.791533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eef3fe-9945-459f-8add-8bd8e91fe0c8 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:54.795117 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eef3fe-9945-459f-8add-8bd8e91fe0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:54.795117 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eef3fe-9945-459f-8add-8bd8e91fe0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:54.817467 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:54.827395 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-48eef3fe-9945-459f-8add-8bd8e91fe0c8 None None] MFA Rules not processed for user `198fc58f13c84db28ec8c69b1f184c83`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:54.828721 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-48eef3fe-9945-459f-8add-8bd8e91fe0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:54.912610 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1110/2232] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:27:54.918280 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3d4e3-f925-4f42-b810-795865c3e12c admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:55.005538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3d4e3-f925-4f42-b810-795865c3e12c admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:55.011304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470655a-62f7-4e93-8d38-5555245eb04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:55.011869 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470655a-62f7-4e93-8d38-5555245eb04e admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:55.012145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470655a-62f7-4e93-8d38-5555245eb04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:55.012425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470655a-62f7-4e93-8d38-5555245eb04e admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:55.013081 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470655a-62f7-4e93-8d38-5555245eb04e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:55.015029 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470655a-62f7-4e93-8d38-5555245eb04e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:55.140749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1123/2233] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:54 2022] POST /identity/v3/projects => generated 381 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:27:55.146041 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eef3fe-9945-459f-8add-8bd8e91fe0c8 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:55.165507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eef3fe-9945-459f-8add-8bd8e91fe0c8 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:55.251992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4bde3a-f933-43de-bf7f-0c779c5af3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:55.252960 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bde3a-f933-43de-bf7f-0c779c5af3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:55.253424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bde3a-f933-43de-bf7f-0c779c5af3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:55.253754 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bde3a-f933-43de-bf7f-0c779c5af3c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:55.254399 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4bde3a-f933-43de-bf7f-0c779c5af3c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:55.256434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4bde3a-f933-43de-bf7f-0c779c5af3c2 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:55.385016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1111/2234] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:55 2022] POST /identity/v3/users => generated 297 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:55.386381 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470655a-62f7-4e93-8d38-5555245eb04e admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:55.398308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470655a-62f7-4e93-8d38-5555245eb04e admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:55.403907 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657f5e7-8910-41fe-8b50-d71d84aaf594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:55.404530 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657f5e7-8910-41fe-8b50-d71d84aaf594 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:55.404814 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657f5e7-8910-41fe-8b50-d71d84aaf594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:55.405090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657f5e7-8910-41fe-8b50-d71d84aaf594 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:55.405745 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657f5e7-8910-41fe-8b50-d71d84aaf594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:55.407514 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657f5e7-8910-41fe-8b50-d71d84aaf594 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:55.465839 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1124/2235] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:55.472316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4bde3a-f933-43de-bf7f-0c779c5af3c2 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:55.486523 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4bde3a-f933-43de-bf7f-0c779c5af3c2 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:55.492138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970773d-c094-4b7d-bc26-8f6a00489dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:55.492859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970773d-c094-4b7d-bc26-8f6a00489dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:55.493138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970773d-c094-4b7d-bc26-8f6a00489dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:55.493720 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970773d-c094-4b7d-bc26-8f6a00489dc4 admin admin] PATH_INFO: `/v3/projects/15bbdb859d63493b9ff8e106cc904537/users/95d8300936314e52aa39814eab3b4b69/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:55.512211 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970773d-c094-4b7d-bc26-8f6a00489dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15bbdb859d63493b9ff8e106cc904537, user_id=95d8300936314e52aa39814eab3b4b69, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:55.514937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970773d-c094-4b7d-bc26-8f6a00489dc4 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:55.621382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1112/2236] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:55 2022] PUT /identity/v3/projects/15bbdb859d63493b9ff8e106cc904537/users/95d8300936314e52aa39814eab3b4b69/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:55.625369 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657f5e7-8910-41fe-8b50-d71d84aaf594 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:55.636732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657f5e7-8910-41fe-8b50-d71d84aaf594 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:55.701369 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743a34e-65bd-4256-a296-4633a77da944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:55.701983 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743a34e-65bd-4256-a296-4633a77da944 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:55.702261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743a34e-65bd-4256-a296-4633a77da944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:55.702532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743a34e-65bd-4256-a296-4633a77da944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:55.703214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743a34e-65bd-4256-a296-4633a77da944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:55.704818 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743a34e-65bd-4256-a296-4633a77da944 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:55.718569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1125/2237] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:55.719778 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970773d-c094-4b7d-bc26-8f6a00489dc4 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:55.734892 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970773d-c094-4b7d-bc26-8f6a00489dc4 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:55.737497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a295c97-8460-4951-8aee-47b4f5f96a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:55.738164 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a295c97-8460-4951-8aee-47b4f5f96a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:55.738450 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a295c97-8460-4951-8aee-47b4f5f96a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:55.738734 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a295c97-8460-4951-8aee-47b4f5f96a6d admin admin] PATH_INFO: `/v3/projects/15bbdb859d63493b9ff8e106cc904537/users/95d8300936314e52aa39814eab3b4b69/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:55.740812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a295c97-8460-4951-8aee-47b4f5f96a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=15bbdb859d63493b9ff8e106cc904537, user_id=95d8300936314e52aa39814eab3b4b69, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:55.744604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a295c97-8460-4951-8aee-47b4f5f96a6d admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:55.755973 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1113/2238] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:55 2022] PUT /identity/v3/projects/15bbdb859d63493b9ff8e106cc904537/users/95d8300936314e52aa39814eab3b4b69/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:55.768142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcacda7-0524-4af3-addf-0819c6c7b799 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:55.768142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcacda7-0524-4af3-addf-0819c6c7b799 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:55.768412 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcacda7-0524-4af3-addf-0819c6c7b799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:55.798959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:55.801001 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-ffcacda7-0524-4af3-addf-0819c6c7b799 None None] MFA Rules not processed for user `95d8300936314e52aa39814eab3b4b69`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:55.805239 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ffcacda7-0524-4af3-addf-0819c6c7b799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:55.891306 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1126/2239] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:55.902619 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a295c97-8460-4951-8aee-47b4f5f96a6d admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:55.904507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8a295c97-8460-4951-8aee-47b4f5f96a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:55.980695 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a295c97-8460-4951-8aee-47b4f5f96a6d admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:55.984591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1afe5c-66c9-4137-94f0-86710caaaea2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:55.985238 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1afe5c-66c9-4137-94f0-86710caaaea2 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:55.985523 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1afe5c-66c9-4137-94f0-86710caaaea2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:55.985813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1afe5c-66c9-4137-94f0-86710caaaea2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:55.995781 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:55.997065 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1afe5c-66c9-4137-94f0-86710caaaea2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:55.998775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1afe5c-66c9-4137-94f0-86710caaaea2 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:56.009575 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1114/2240] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:56.642235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcacda7-0524-4af3-addf-0819c6c7b799 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:56.721618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcacda7-0524-4af3-addf-0819c6c7b799 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:56.726619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c09d64-f9a7-4fe4-ac2c-50cca8b5e4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:56.727288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c09d64-f9a7-4fe4-ac2c-50cca8b5e4d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:56.727760 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c09d64-f9a7-4fe4-ac2c-50cca8b5e4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:56.728042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c09d64-f9a7-4fe4-ac2c-50cca8b5e4d4 admin admin] PATH_INFO: `/v3/users/198fc58f13c84db28ec8c69b1f184c83` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:56.729232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c09d64-f9a7-4fe4-ac2c-50cca8b5e4d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=198fc58f13c84db28ec8c69b1f184c83)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:56.731293 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c09d64-f9a7-4fe4-ac2c-50cca8b5e4d4 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:56.993860 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e9c09d64-f9a7-4fe4-ac2c-50cca8b5e4d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '198fc58f13c84db28ec8c69b1f184c83'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:57.008201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e9c09d64-f9a7-4fe4-ac2c-50cca8b5e4d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '198fc58f13c84db28ec8c69b1f184c83'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:57.105953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e9c09d64-f9a7-4fe4-ac2c-50cca8b5e4d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '198fc58f13c84db28ec8c69b1f184c83'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:57.116891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e9c09d64-f9a7-4fe4-ac2c-50cca8b5e4d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '198fc58f13c84db28ec8c69b1f184c83'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:57.118188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e9c09d64-f9a7-4fe4-ac2c-50cca8b5e4d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '198fc58f13c84db28ec8c69b1f184c83'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:57.124404 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1127/2241] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:56 2022] DELETE /identity/v3/users/198fc58f13c84db28ec8c69b1f184c83 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:57.128789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc5678-8bdd-414c-aff4-55c150da0c83 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:57.129252 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc5678-8bdd-414c-aff4-55c150da0c83 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:57.129346 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc5678-8bdd-414c-aff4-55c150da0c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:57.149599 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-fddc5678-8bdd-414c-aff4-55c150da0c83 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:57.150951 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-fddc5678-8bdd-414c-aff4-55c150da0c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:57.218627 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1115/2242] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:57.229684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c09d64-f9a7-4fe4-ac2c-50cca8b5e4d4 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:57.231154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e9c09d64-f9a7-4fe4-ac2c-50cca8b5e4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:57.253555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:57.301687 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c09d64-f9a7-4fe4-ac2c-50cca8b5e4d4 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:57.307148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b1a77-27e7-4c4e-9e40-7e5368690586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:57.308412 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b1a77-27e7-4c4e-9e40-7e5368690586 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:57.308700 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b1a77-27e7-4c4e-9e40-7e5368690586 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:57.308975 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b1a77-27e7-4c4e-9e40-7e5368690586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:57.318689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b1a77-27e7-4c4e-9e40-7e5368690586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:57.320148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b1a77-27e7-4c4e-9e40-7e5368690586 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:57.330573 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1128/2243] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:57.829853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc5678-8bdd-414c-aff4-55c150da0c83 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:57.831341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-fddc5678-8bdd-414c-aff4-55c150da0c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:57.844396 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc5678-8bdd-414c-aff4-55c150da0c83 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:57.850350 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bc35a-f945-438d-967e-d09317b876f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:57.851192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bc35a-f945-438d-967e-d09317b876f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:57.851451 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bc35a-f945-438d-967e-d09317b876f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:57.851741 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bc35a-f945-438d-967e-d09317b876f0 admin admin] PATH_INFO: `/v3/projects/0560bd88f0fe475cb6bc8980323943f7` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:57.853546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73bc35a-f945-438d-967e-d09317b876f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0560bd88f0fe475cb6bc8980323943f7)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:57.855503 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73bc35a-f945-438d-967e-d09317b876f0 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:58.021978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-b73bc35a-f945-438d-967e-d09317b876f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0560bd88f0fe475cb6bc8980323943f7', 'request_id': 'req-b73bc35a-f945-438d-967e-d09317b876f0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:58.030335 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-b73bc35a-f945-438d-967e-d09317b876f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0560bd88f0fe475cb6bc8980323943f7', 'request_id': 'req-b73bc35a-f945-438d-967e-d09317b876f0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:58.036788 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-b73bc35a-f945-438d-967e-d09317b876f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0560bd88f0fe475cb6bc8980323943f7', 'request_id': 'req-b73bc35a-f945-438d-967e-d09317b876f0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:58.137295 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-b73bc35a-f945-438d-967e-d09317b876f0 admin admin] The token cache is being invalidate because project 0560bd88f0fe475cb6bc8980323943f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:58.137981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-b73bc35a-f945-438d-967e-d09317b876f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:58.138380 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-b73bc35a-f945-438d-967e-d09317b876f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:58.140177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1116/2244] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:57 2022] DELETE /identity/v3/projects/0560bd88f0fe475cb6bc8980323943f7 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:58.144142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b1a77-27e7-4c4e-9e40-7e5368690586 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:58.155108 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b1a77-27e7-4c4e-9e40-7e5368690586 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:58.217683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068bb3d-5a4a-4214-9ca6-c2f8b8e5149f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:58.218243 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068bb3d-5a4a-4214-9ca6-c2f8b8e5149f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:58.218523 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068bb3d-5a4a-4214-9ca6-c2f8b8e5149f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:58.218803 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068bb3d-5a4a-4214-9ca6-c2f8b8e5149f admin admin] PATH_INFO: `/v3/users/95d8300936314e52aa39814eab3b4b69` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:58.220049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068bb3d-5a4a-4214-9ca6-c2f8b8e5149f admin admin] RBAC: Authorizing `identity:delete_user(user_id=95d8300936314e52aa39814eab3b4b69)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:58.225287 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068bb3d-5a4a-4214-9ca6-c2f8b8e5149f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:58.437890 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f068bb3d-5a4a-4214-9ca6-c2f8b8e5149f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95d8300936314e52aa39814eab3b4b69'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:58.450678 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f068bb3d-5a4a-4214-9ca6-c2f8b8e5149f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95d8300936314e52aa39814eab3b4b69'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:58.521017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f068bb3d-5a4a-4214-9ca6-c2f8b8e5149f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95d8300936314e52aa39814eab3b4b69'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:58.528833 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f068bb3d-5a4a-4214-9ca6-c2f8b8e5149f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95d8300936314e52aa39814eab3b4b69'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:58.529530 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f068bb3d-5a4a-4214-9ca6-c2f8b8e5149f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95d8300936314e52aa39814eab3b4b69'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:58.535829 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1129/2245] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:58 2022] DELETE /identity/v3/users/95d8300936314e52aa39814eab3b4b69 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:58.987004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bc35a-f945-438d-967e-d09317b876f0 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:58.988357 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b73bc35a-f945-438d-967e-d09317b876f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:59.002166 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bc35a-f945-438d-967e-d09317b876f0 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:59.063043 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09923c-ce14-47c8-80a0-c991c24eabe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:59.063680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09923c-ce14-47c8-80a0-c991c24eabe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:59.063963 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09923c-ce14-47c8-80a0-c991c24eabe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:59.064259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09923c-ce14-47c8-80a0-c991c24eabe4 admin admin] PATH_INFO: `/v3/projects/15bbdb859d63493b9ff8e106cc904537` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:59.065370 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09923c-ce14-47c8-80a0-c991c24eabe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15bbdb859d63493b9ff8e106cc904537)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:59.067342 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09923c-ce14-47c8-80a0-c991c24eabe4 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:59.090748 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:59.217659 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7b09923c-ce14-47c8-80a0-c991c24eabe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15bbdb859d63493b9ff8e106cc904537', 'request_id': 'req-7b09923c-ce14-47c8-80a0-c991c24eabe4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:59.223597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7b09923c-ce14-47c8-80a0-c991c24eabe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15bbdb859d63493b9ff8e106cc904537', 'request_id': 'req-7b09923c-ce14-47c8-80a0-c991c24eabe4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:59.231331 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7b09923c-ce14-47c8-80a0-c991c24eabe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15bbdb859d63493b9ff8e106cc904537', 'request_id': 'req-7b09923c-ce14-47c8-80a0-c991c24eabe4'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:59.303186 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068bb3d-5a4a-4214-9ca6-c2f8b8e5149f admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:59.305017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f068bb3d-5a4a-4214-9ca6-c2f8b8e5149f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:59.317469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068bb3d-5a4a-4214-9ca6-c2f8b8e5149f admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:59.365365 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:59.384956 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ef16f-4eaa-43d0-a2c2-414bb834f54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:59.385669 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ef16f-4eaa-43d0-a2c2-414bb834f54e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:59.385945 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ef16f-4eaa-43d0-a2c2-414bb834f54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:59.386216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ef16f-4eaa-43d0-a2c2-414bb834f54e admin admin] PATH_INFO: `/v3/users/1cc4dfb3be554d6e8b419a01f7f608c4` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:59.387368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ef16f-4eaa-43d0-a2c2-414bb834f54e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cc4dfb3be554d6e8b419a01f7f608c4)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:59.389270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ef16f-4eaa-43d0-a2c2-414bb834f54e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:59.427119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7b09923c-ce14-47c8-80a0-c991c24eabe4 admin admin] The token cache is being invalidate because project 15bbdb859d63493b9ff8e106cc904537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:59.427415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7b09923c-ce14-47c8-80a0-c991c24eabe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:59.428047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7b09923c-ce14-47c8-80a0-c991c24eabe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:59.429536 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1117/2246] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:58 2022] DELETE /identity/v3/projects/15bbdb859d63493b9ff8e106cc904537 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:59.434463 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56a70e-3b90-4405-8261-a1bcb626aa6a None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:59.434745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56a70e-3b90-4405-8261-a1bcb626aa6a None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:59.435064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56a70e-3b90-4405-8261-a1bcb626aa6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:59.455497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-1e56a70e-3b90-4405-8261-a1bcb626aa6a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:59.456768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-1e56a70e-3b90-4405-8261-a1bcb626aa6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:59.465883 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1118/2247] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:27:59.473565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8adf16-6d5f-499a-9df4-bd4ce2ea9c23 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:59.473854 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8adf16-6d5f-499a-9df4-bd4ce2ea9c23 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:59.474131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8adf16-6d5f-499a-9df4-bd4ce2ea9c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:59.494199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ab8adf16-6d5f-499a-9df4-bd4ce2ea9c23 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:59.495520 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ab8adf16-6d5f-499a-9df4-bd4ce2ea9c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:59.504165 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1119/2248] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:27:59.507986 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8adf16-6d5f-499a-9df4-bd4ce2ea9c23 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:59.525370 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8adf16-6d5f-499a-9df4-bd4ce2ea9c23 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:59.525370 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a98a4a-977a-42a0-a798-7d0b915f5e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:59.525370 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a98a4a-977a-42a0-a798-7d0b915f5e5c admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:59.525370 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a98a4a-977a-42a0-a798-7d0b915f5e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:59.525967 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a98a4a-977a-42a0-a798-7d0b915f5e5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:59.525967 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a98a4a-977a-42a0-a798-7d0b915f5e5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:27:59.525967 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a98a4a-977a-42a0-a798-7d0b915f5e5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:59.527159 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a98a4a-977a-42a0-a798-7d0b915f5e5c admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:59.540127 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1120/2249] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:27:59.544241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a98a4a-977a-42a0-a798-7d0b915f5e5c admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:59.557899 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a98a4a-977a-42a0-a798-7d0b915f5e5c admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:59.563468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d2d6a-8359-44f4-8d0c-b9c78f6375f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:59.564068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d2d6a-8359-44f4-8d0c-b9c78f6375f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:59.564349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d2d6a-8359-44f4-8d0c-b9c78f6375f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:59.564648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d2d6a-8359-44f4-8d0c-b9c78f6375f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:59.565278 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d2d6a-8359-44f4-8d0c-b9c78f6375f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:59.567335 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d2d6a-8359-44f4-8d0c-b9c78f6375f1 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:59.645019 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2b1ef16f-4eaa-43d0-a2c2-414bb834f54e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cc4dfb3be554d6e8b419a01f7f608c4'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:59.655321 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1121/2250] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:59 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:59.659715 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d2d6a-8359-44f4-8d0c-b9c78f6375f1 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:59.664486 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2b1ef16f-4eaa-43d0-a2c2-414bb834f54e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cc4dfb3be554d6e8b419a01f7f608c4'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:59.734205 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d2d6a-8359-44f4-8d0c-b9c78f6375f1 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:59.739962 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd524ef-007c-4512-b74a-96277ef51141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:59.740606 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd524ef-007c-4512-b74a-96277ef51141 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:59.740917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd524ef-007c-4512-b74a-96277ef51141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:59.741176 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd524ef-007c-4512-b74a-96277ef51141 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:59.741947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd524ef-007c-4512-b74a-96277ef51141 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:59.745306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd524ef-007c-4512-b74a-96277ef51141 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:59.770244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2b1ef16f-4eaa-43d0-a2c2-414bb834f54e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cc4dfb3be554d6e8b419a01f7f608c4'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:59.778804 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2b1ef16f-4eaa-43d0-a2c2-414bb834f54e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cc4dfb3be554d6e8b419a01f7f608c4'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:59.779584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2b1ef16f-4eaa-43d0-a2c2-414bb834f54e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cc4dfb3be554d6e8b419a01f7f608c4'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:59.785462 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1130/2251] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:59 2022] DELETE /identity/v3/users/1cc4dfb3be554d6e8b419a01f7f608c4 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:27:59.789865 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b155c9-f3ff-462c-9f56-97e617c7afdf None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:59.790147 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b155c9-f3ff-462c-9f56-97e617c7afdf None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:59.790496 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b155c9-f3ff-462c-9f56-97e617c7afdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:59.809338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-80b155c9-f3ff-462c-9f56-97e617c7afdf None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:59.810570 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-80b155c9-f3ff-462c-9f56-97e617c7afdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:59.828440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1122/2252] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:59 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:27:59.829535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd524ef-007c-4512-b74a-96277ef51141 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:59.831196 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-1fd524ef-007c-4512-b74a-96277ef51141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:59.843416 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd524ef-007c-4512-b74a-96277ef51141 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:59.881785 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1131/2253] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:27:59.896085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b155c9-f3ff-462c-9f56-97e617c7afdf None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:59.899164 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-80b155c9-f3ff-462c-9f56-97e617c7afdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:59.910549 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9263dc-d03c-4d56-9c13-0ab6c59fa585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:59.911283 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9263dc-d03c-4d56-9c13-0ab6c59fa585 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:59.911629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9263dc-d03c-4d56-9c13-0ab6c59fa585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:59.911968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9263dc-d03c-4d56-9c13-0ab6c59fa585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:59.912677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9263dc-d03c-4d56-9c13-0ab6c59fa585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:59.914381 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9263dc-d03c-4d56-9c13-0ab6c59fa585 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:59.924821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1123/2254] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:27:59.928933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9263dc-d03c-4d56-9c13-0ab6c59fa585 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:59.939140 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9263dc-d03c-4d56-9c13-0ab6c59fa585 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:59.943853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5c5f8-92f7-45d7-8573-d6d7b764210b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:59.944449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5c5f8-92f7-45d7-8573-d6d7b764210b admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:59.944728 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5c5f8-92f7-45d7-8573-d6d7b764210b admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:59.944998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5c5f8-92f7-45d7-8573-d6d7b764210b admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:59.945631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5c5f8-92f7-45d7-8573-d6d7b764210b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:59.947894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5c5f8-92f7-45d7-8573-d6d7b764210b admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:59.958712 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1124/2255] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:27:59.958865 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b155c9-f3ff-462c-9f56-97e617c7afdf None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:59.962780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef41817-fa18-4423-80d6-abbd30136495 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:59.963154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5c5f8-92f7-45d7-8573-d6d7b764210b admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:59.963427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef41817-fa18-4423-80d6-abbd30136495 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:59.963836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef41817-fa18-4423-80d6-abbd30136495 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:59.964117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef41817-fa18-4423-80d6-abbd30136495 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:59.972794 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5c5f8-92f7-45d7-8573-d6d7b764210b admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:59.974044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef41817-fa18-4423-80d6-abbd30136495 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:59.975711 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef41817-fa18-4423-80d6-abbd30136495 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:59.977877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5f326-83d8-4d3b-a9f8-665c77fa96f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:59.978566 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5f326-83d8-4d3b-a9f8-665c77fa96f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:59.978845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5f326-83d8-4d3b-a9f8-665c77fa96f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:59.979158 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5f326-83d8-4d3b-a9f8-665c77fa96f5 admin admin] PATH_INFO: `/v3/projects/bf0c7660e1db46f38b024ab7b5a0a5c4/users/04932b08ed4a4862824329c68550c2df/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:59.988267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1132/2256] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:59.996292 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5f326-83d8-4d3b-a9f8-665c77fa96f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf0c7660e1db46f38b024ab7b5a0a5c4, user_id=04932b08ed4a4862824329c68550c2df, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:59.999495 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5f326-83d8-4d3b-a9f8-665c77fa96f5 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:00.069153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1125/2257] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:59 2022] PUT /identity/v3/projects/bf0c7660e1db46f38b024ab7b5a0a5c4/users/04932b08ed4a4862824329c68550c2df/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:28:00.081027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136acffa-b656-4c9a-968f-1ad88496a1e3 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:00.081339 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136acffa-b656-4c9a-968f-1ad88496a1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:00.081615 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136acffa-b656-4c9a-968f-1ad88496a1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:00.112399 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-136acffa-b656-4c9a-968f-1ad88496a1e3 None None] MFA Rules not processed for user `04932b08ed4a4862824329c68550c2df`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:00.113598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-136acffa-b656-4c9a-968f-1ad88496a1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:00.186411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1133/2258] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:00.192920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5f326-83d8-4d3b-a9f8-665c77fa96f5 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:00.194702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c6d5f326-83d8-4d3b-a9f8-665c77fa96f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:00.272453 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:00.279864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5f326-83d8-4d3b-a9f8-665c77fa96f5 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:00.286264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03be40f-15ae-4844-9c02-79c6b65a8e3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:00.287351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03be40f-15ae-4844-9c02-79c6b65a8e3c service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:00.287871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03be40f-15ae-4844-9c02-79c6b65a8e3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:00.288368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03be40f-15ae-4844-9c02-79c6b65a8e3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:00.304656 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03be40f-15ae-4844-9c02-79c6b65a8e3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:00.311386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03be40f-15ae-4844-9c02-79c6b65a8e3c service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:00.317679 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1126/2259] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:00.343289 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136acffa-b656-4c9a-968f-1ad88496a1e3 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:00.361763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:00.436316 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136acffa-b656-4c9a-968f-1ad88496a1e3 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:00.441622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c1599-1fcd-4f68-a1c4-4d49d7e181be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:00.442260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c1599-1fcd-4f68-a1c4-4d49d7e181be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:00.442582 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c1599-1fcd-4f68-a1c4-4d49d7e181be admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:00.442948 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c1599-1fcd-4f68-a1c4-4d49d7e181be admin admin] PATH_INFO: `/v3/users/04932b08ed4a4862824329c68550c2df` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:00.444106 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c1599-1fcd-4f68-a1c4-4d49d7e181be admin admin] RBAC: Authorizing `identity:delete_user(user_id=04932b08ed4a4862824329c68550c2df)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:00.447129 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c1599-1fcd-4f68-a1c4-4d49d7e181be admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:00.587456 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03be40f-15ae-4844-9c02-79c6b65a8e3c service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:00.588971 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c03be40f-15ae-4844-9c02-79c6b65a8e3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:00.605113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03be40f-15ae-4844-9c02-79c6b65a8e3c service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:00.610210 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b00dc-921a-42c4-bb49-53c397cc6842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:00.621984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b00dc-921a-42c4-bb49-53c397cc6842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:00.621984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b00dc-921a-42c4-bb49-53c397cc6842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:00.621984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b00dc-921a-42c4-bb49-53c397cc6842 admin admin] PATH_INFO: `/v3/projects/7f1096f5ccd9443192c4d91d8984a633` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:00.621984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b00dc-921a-42c4-bb49-53c397cc6842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f1096f5ccd9443192c4d91d8984a633)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:00.621984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b00dc-921a-42c4-bb49-53c397cc6842 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:00.706621 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-cf2c1599-1fcd-4f68-a1c4-4d49d7e181be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04932b08ed4a4862824329c68550c2df'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:00.725836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-cf2c1599-1fcd-4f68-a1c4-4d49d7e181be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04932b08ed4a4862824329c68550c2df'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:00.750607 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-746b00dc-921a-42c4-bb49-53c397cc6842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f1096f5ccd9443192c4d91d8984a633', 'request_id': 'req-746b00dc-921a-42c4-bb49-53c397cc6842'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:00.756139 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-746b00dc-921a-42c4-bb49-53c397cc6842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f1096f5ccd9443192c4d91d8984a633', 'request_id': 'req-746b00dc-921a-42c4-bb49-53c397cc6842'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:00.762983 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-746b00dc-921a-42c4-bb49-53c397cc6842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f1096f5ccd9443192c4d91d8984a633', 'request_id': 'req-746b00dc-921a-42c4-bb49-53c397cc6842'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:00.835588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-cf2c1599-1fcd-4f68-a1c4-4d49d7e181be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04932b08ed4a4862824329c68550c2df'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:00.843349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-cf2c1599-1fcd-4f68-a1c4-4d49d7e181be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04932b08ed4a4862824329c68550c2df'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:00.843997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-cf2c1599-1fcd-4f68-a1c4-4d49d7e181be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04932b08ed4a4862824329c68550c2df'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:00.850275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1134/2260] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:00 2022] DELETE /identity/v3/users/04932b08ed4a4862824329c68550c2df => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:00.855013 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01123f8-bc2e-4f8c-a43f-5088c6a660d3 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:00.855287 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01123f8-bc2e-4f8c-a43f-5088c6a660d3 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:00.855561 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01123f8-bc2e-4f8c-a43f-5088c6a660d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:00.876429 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d01123f8-bc2e-4f8c-a43f-5088c6a660d3 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:00.877676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d01123f8-bc2e-4f8c-a43f-5088c6a660d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:00.934580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-746b00dc-921a-42c4-bb49-53c397cc6842 admin admin] The token cache is being invalidate because project 7f1096f5ccd9443192c4d91d8984a633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:00.934580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-746b00dc-921a-42c4-bb49-53c397cc6842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:00.935371 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-746b00dc-921a-42c4-bb49-53c397cc6842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:00.937201 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1127/2261] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:00 2022] DELETE /identity/v3/projects/7f1096f5ccd9443192c4d91d8984a633 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:28:00.947504 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1135/2262] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:28:00.955090 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b00dc-921a-42c4-bb49-53c397cc6842 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:00.956733 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-746b00dc-921a-42c4-bb49-53c397cc6842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:01.024865 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b00dc-921a-42c4-bb49-53c397cc6842 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:01.120566 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d647c5-83ca-4488-8eb8-6cb6f5cc764a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:01.121241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d647c5-83ca-4488-8eb8-6cb6f5cc764a service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:01.121524 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d647c5-83ca-4488-8eb8-6cb6f5cc764a service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:01.121798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d647c5-83ca-4488-8eb8-6cb6f5cc764a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:01.132225 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d647c5-83ca-4488-8eb8-6cb6f5cc764a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:01.133723 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d647c5-83ca-4488-8eb8-6cb6f5cc764a service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:01.144403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1128/2263] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:01.502138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:01.632646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:01.784432 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01123f8-bc2e-4f8c-a43f-5088c6a660d3 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:01.785849 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d01123f8-bc2e-4f8c-a43f-5088c6a660d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:01.797564 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01123f8-bc2e-4f8c-a43f-5088c6a660d3 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:01.802156 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1881e1-89e4-4ee6-aaa8-c6b85e57ee6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:01.802968 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1881e1-89e4-4ee6-aaa8-c6b85e57ee6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:01.803092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1881e1-89e4-4ee6-aaa8-c6b85e57ee6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:01.803339 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1881e1-89e4-4ee6-aaa8-c6b85e57ee6e admin admin] PATH_INFO: `/v3/projects/bf0c7660e1db46f38b024ab7b5a0a5c4` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:01.804556 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1881e1-89e4-4ee6-aaa8-c6b85e57ee6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf0c7660e1db46f38b024ab7b5a0a5c4)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:01.806513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1881e1-89e4-4ee6-aaa8-c6b85e57ee6e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:01.974458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-cc1881e1-89e4-4ee6-aaa8-c6b85e57ee6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf0c7660e1db46f38b024ab7b5a0a5c4', 'request_id': 'req-cc1881e1-89e4-4ee6-aaa8-c6b85e57ee6e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:01.980373 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-cc1881e1-89e4-4ee6-aaa8-c6b85e57ee6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf0c7660e1db46f38b024ab7b5a0a5c4', 'request_id': 'req-cc1881e1-89e4-4ee6-aaa8-c6b85e57ee6e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:02.092983 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-cc1881e1-89e4-4ee6-aaa8-c6b85e57ee6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf0c7660e1db46f38b024ab7b5a0a5c4', 'request_id': 'req-cc1881e1-89e4-4ee6-aaa8-c6b85e57ee6e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:02.107695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-cc1881e1-89e4-4ee6-aaa8-c6b85e57ee6e admin admin] The token cache is being invalidate because project bf0c7660e1db46f38b024ab7b5a0a5c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:02.108012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-cc1881e1-89e4-4ee6-aaa8-c6b85e57ee6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:02.108664 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-cc1881e1-89e4-4ee6-aaa8-c6b85e57ee6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:02.110210 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1136/2264] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:01 2022] DELETE /identity/v3/projects/bf0c7660e1db46f38b024ab7b5a0a5c4 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:02.115355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7a287-1f7c-418e-8f56-b751f3f0b123 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.115693 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7a287-1f7c-418e-8f56-b751f3f0b123 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.115964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7a287-1f7c-418e-8f56-b751f3f0b123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.135612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-3fb7a287-1f7c-418e-8f56-b751f3f0b123 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:02.137064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-3fb7a287-1f7c-418e-8f56-b751f3f0b123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:02.197909 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1129/2265] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:02.205569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebdce7-8441-4ded-8d48-f650c30c854f None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.205854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebdce7-8441-4ded-8d48-f650c30c854f None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.206130 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebdce7-8441-4ded-8d48-f650c30c854f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.229874 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-fdebdce7-8441-4ded-8d48-f650c30c854f None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:02.231048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-fdebdce7-8441-4ded-8d48-f650c30c854f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:02.239843 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1137/2266] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:02.244300 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7a287-1f7c-418e-8f56-b751f3f0b123 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:02.254142 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7a287-1f7c-418e-8f56-b751f3f0b123 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:02.259137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7353f-a4f4-4f59-a0c6-606d4002827a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:02.259875 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7353f-a4f4-4f59-a0c6-606d4002827a admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.260249 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7353f-a4f4-4f59-a0c6-606d4002827a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.260525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7353f-a4f4-4f59-a0c6-606d4002827a admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.261173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7353f-a4f4-4f59-a0c6-606d4002827a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:28:02.261483 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7353f-a4f4-4f59-a0c6-606d4002827a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:02.263129 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7353f-a4f4-4f59-a0c6-606d4002827a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:02.279906 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1130/2267] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:28:02.281292 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebdce7-8441-4ded-8d48-f650c30c854f None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:02.294059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebdce7-8441-4ded-8d48-f650c30c854f None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:02.299670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934152a-4f20-43c9-8bb0-85d5e7260c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:02.300308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934152a-4f20-43c9-8bb0-85d5e7260c70 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.300750 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934152a-4f20-43c9-8bb0-85d5e7260c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.301037 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934152a-4f20-43c9-8bb0-85d5e7260c70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.301689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934152a-4f20-43c9-8bb0-85d5e7260c70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:02.303705 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934152a-4f20-43c9-8bb0-85d5e7260c70 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:02.407569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1138/2268] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:28:02 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:02.411650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7353f-a4f4-4f59-a0c6-606d4002827a admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:02.422005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7353f-a4f4-4f59-a0c6-606d4002827a admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:02.479535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb47ee-cfca-46a7-a6c6-b48745d4f5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:02.480170 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb47ee-cfca-46a7-a6c6-b48745d4f5d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.480454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb47ee-cfca-46a7-a6c6-b48745d4f5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.480756 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb47ee-cfca-46a7-a6c6-b48745d4f5d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.481424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb47ee-cfca-46a7-a6c6-b48745d4f5d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:02.483557 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb47ee-cfca-46a7-a6c6-b48745d4f5d7 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:02.591051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1131/2269] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:28:02 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:02.595346 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934152a-4f20-43c9-8bb0-85d5e7260c70 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:02.606281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934152a-4f20-43c9-8bb0-85d5e7260c70 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:02.611155 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61087425-3e0a-4178-950e-96a6a66be6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:02.619807 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61087425-3e0a-4178-950e-96a6a66be6ea admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.619807 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61087425-3e0a-4178-950e-96a6a66be6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.619807 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61087425-3e0a-4178-950e-96a6a66be6ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.619807 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61087425-3e0a-4178-950e-96a6a66be6ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:02.619807 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61087425-3e0a-4178-950e-96a6a66be6ea admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:02.626826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128c9d3-d041-4b29-a0bb-b19ea7150cd2 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.627141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128c9d3-d041-4b29-a0bb-b19ea7150cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.627475 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128c9d3-d041-4b29-a0bb-b19ea7150cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.629246 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1139/2270] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:02.632911 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61087425-3e0a-4178-950e-96a6a66be6ea admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:02.643492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61087425-3e0a-4178-950e-96a6a66be6ea admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:02.647821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-c128c9d3-d041-4b29-a0bb-b19ea7150cd2 None None] MFA Rules not processed for user `d441bbdb43c5403abf8480146bc15753`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:02.648276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0f106-8cc7-4870-afed-d0d942ea4271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:02.649582 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0f106-8cc7-4870-afed-d0d942ea4271 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.649806 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c128c9d3-d041-4b29-a0bb-b19ea7150cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:02.649909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0f106-8cc7-4870-afed-d0d942ea4271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.650162 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0f106-8cc7-4870-afed-d0d942ea4271 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.651023 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd0f106-8cc7-4870-afed-d0d942ea4271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:02.652602 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd0f106-8cc7-4870-afed-d0d942ea4271 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:02.663156 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1140/2271] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:28:02.667360 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0f106-8cc7-4870-afed-d0d942ea4271 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:02.677096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0f106-8cc7-4870-afed-d0d942ea4271 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:02.683283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa349c-4561-4195-90ef-838b0b43813d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:02.683857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa349c-4561-4195-90ef-838b0b43813d admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.684137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa349c-4561-4195-90ef-838b0b43813d admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.684411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa349c-4561-4195-90ef-838b0b43813d admin admin] PATH_INFO: `/v3/projects/26bf01f85f54495199e9b9141064594c/users/127d83c8f4dc4f559d9ce32435577a7c/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.704417 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa349c-4561-4195-90ef-838b0b43813d admin admin] RBAC: Authorizing `identity:create_grant(project_id=26bf01f85f54495199e9b9141064594c, user_id=127d83c8f4dc4f559d9ce32435577a7c, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:02.708044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa349c-4561-4195-90ef-838b0b43813d admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:02.717706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1132/2272] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:02 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:02.813668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1141/2273] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:02 2022] PUT /identity/v3/projects/26bf01f85f54495199e9b9141064594c/users/127d83c8f4dc4f559d9ce32435577a7c/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:28:02.824811 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86fc34-7381-497a-868a-c3be700c4e02 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.825103 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86fc34-7381-497a-868a-c3be700c4e02 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.825378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86fc34-7381-497a-868a-c3be700c4e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.859048 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-8a86fc34-7381-497a-868a-c3be700c4e02 None None] MFA Rules not processed for user `127d83c8f4dc4f559d9ce32435577a7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:02.860439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8a86fc34-7381-497a-868a-c3be700c4e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:02.945040 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1133/2274] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:28:02.957681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa349c-4561-4195-90ef-838b0b43813d admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:02.959546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-aaaa349c-4561-4195-90ef-838b0b43813d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:03.032240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa349c-4561-4195-90ef-838b0b43813d admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:03.036353 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6118af95-3ba9-4f91-94a6-3ed5a48a8b28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:03.036997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118af95-3ba9-4f91-94a6-3ed5a48a8b28 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:03.037276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118af95-3ba9-4f91-94a6-3ed5a48a8b28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:03.037547 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118af95-3ba9-4f91-94a6-3ed5a48a8b28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:03.047106 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6118af95-3ba9-4f91-94a6-3ed5a48a8b28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:03.048577 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6118af95-3ba9-4f91-94a6-3ed5a48a8b28 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:03.060227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1142/2275] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:03.082967 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86fc34-7381-497a-868a-c3be700c4e02 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:03.144721 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:03.169286 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86fc34-7381-497a-868a-c3be700c4e02 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:03.174966 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc76cb-70b1-4fa3-b0f5-ee3164b88a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:03.175957 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc76cb-70b1-4fa3-b0f5-ee3164b88a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:03.176245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc76cb-70b1-4fa3-b0f5-ee3164b88a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:03.176544 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc76cb-70b1-4fa3-b0f5-ee3164b88a79 admin admin] PATH_INFO: `/v3/users/127d83c8f4dc4f559d9ce32435577a7c` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:03.177717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc76cb-70b1-4fa3-b0f5-ee3164b88a79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=127d83c8f4dc4f559d9ce32435577a7c)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:03.180522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc76cb-70b1-4fa3-b0f5-ee3164b88a79 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:03.456324 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d9fc76cb-70b1-4fa3-b0f5-ee3164b88a79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127d83c8f4dc4f559d9ce32435577a7c'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:03.475883 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:03.538197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d9fc76cb-70b1-4fa3-b0f5-ee3164b88a79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127d83c8f4dc4f559d9ce32435577a7c'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:03.539056 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d9fc76cb-70b1-4fa3-b0f5-ee3164b88a79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127d83c8f4dc4f559d9ce32435577a7c'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:03.554917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d9fc76cb-70b1-4fa3-b0f5-ee3164b88a79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127d83c8f4dc4f559d9ce32435577a7c'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:03.556332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d9fc76cb-70b1-4fa3-b0f5-ee3164b88a79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127d83c8f4dc4f559d9ce32435577a7c'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:03.570692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1134/2276] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:03 2022] DELETE /identity/v3/users/127d83c8f4dc4f559d9ce32435577a7c => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:03.576062 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19019d58-8bc7-4aa2-ba79-9c8238aef603 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:03.576355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19019d58-8bc7-4aa2-ba79-9c8238aef603 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:03.576636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19019d58-8bc7-4aa2-ba79-9c8238aef603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:03.609867 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-19019d58-8bc7-4aa2-ba79-9c8238aef603 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:03.611717 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-19019d58-8bc7-4aa2-ba79-9c8238aef603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:03.690019 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1143/2277] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:03.700858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc76cb-70b1-4fa3-b0f5-ee3164b88a79 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:03.702306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d9fc76cb-70b1-4fa3-b0f5-ee3164b88a79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:03.777783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc76cb-70b1-4fa3-b0f5-ee3164b88a79 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:03.785088 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c64a8-a245-4815-83ad-0300c3138692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:03.785727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c64a8-a245-4815-83ad-0300c3138692 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:03.786189 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c64a8-a245-4815-83ad-0300c3138692 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:03.786377 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c64a8-a245-4815-83ad-0300c3138692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:03.796756 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c64a8-a245-4815-83ad-0300c3138692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:03.798556 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c64a8-a245-4815-83ad-0300c3138692 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:03.811820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1135/2278] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:04.283480 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19019d58-8bc7-4aa2-ba79-9c8238aef603 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:04.285037 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-19019d58-8bc7-4aa2-ba79-9c8238aef603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:04.300589 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19019d58-8bc7-4aa2-ba79-9c8238aef603 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:04.304279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:04.305583 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61947f-47e8-4327-bed2-c94a7929b3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:04.306474 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61947f-47e8-4327-bed2-c94a7929b3f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:04.306690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61947f-47e8-4327-bed2-c94a7929b3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:04.306990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61947f-47e8-4327-bed2-c94a7929b3f5 admin admin] PATH_INFO: `/v3/projects/26bf01f85f54495199e9b9141064594c` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:04.308228 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61947f-47e8-4327-bed2-c94a7929b3f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26bf01f85f54495199e9b9141064594c)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:04.310197 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61947f-47e8-4327-bed2-c94a7929b3f5 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:04.442293 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3c61947f-47e8-4327-bed2-c94a7929b3f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26bf01f85f54495199e9b9141064594c', 'request_id': 'req-3c61947f-47e8-4327-bed2-c94a7929b3f5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:04.448285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3c61947f-47e8-4327-bed2-c94a7929b3f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26bf01f85f54495199e9b9141064594c', 'request_id': 'req-3c61947f-47e8-4327-bed2-c94a7929b3f5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:04.521392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3c61947f-47e8-4327-bed2-c94a7929b3f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26bf01f85f54495199e9b9141064594c', 'request_id': 'req-3c61947f-47e8-4327-bed2-c94a7929b3f5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:04.534365 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3c61947f-47e8-4327-bed2-c94a7929b3f5 admin admin] The token cache is being invalidate because project 26bf01f85f54495199e9b9141064594c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:04.534655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3c61947f-47e8-4327-bed2-c94a7929b3f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:04.535324 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3c61947f-47e8-4327-bed2-c94a7929b3f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:04.536824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1144/2279] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:04 2022] DELETE /identity/v3/projects/26bf01f85f54495199e9b9141064594c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:04.542633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fe42d-fbdf-4198-a16a-2149e2ed863b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:04.543044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fe42d-fbdf-4198-a16a-2149e2ed863b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:04.543356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fe42d-fbdf-4198-a16a-2149e2ed863b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:04.563855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ce9fe42d-fbdf-4198-a16a-2149e2ed863b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:04.565125 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ce9fe42d-fbdf-4198-a16a-2149e2ed863b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:04.635627 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1136/2280] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:04.644428 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19470d8-6ed6-4d06-a559-867988a6b67a None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:04.644715 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19470d8-6ed6-4d06-a559-867988a6b67a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:04.644992 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19470d8-6ed6-4d06-a559-867988a6b67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:04.665563 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-c19470d8-6ed6-4d06-a559-867988a6b67a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:04.666771 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c19470d8-6ed6-4d06-a559-867988a6b67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:04.675304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1145/2281] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:04.679423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fe42d-fbdf-4198-a16a-2149e2ed863b None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:04.689212 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fe42d-fbdf-4198-a16a-2149e2ed863b None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:04.693868 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04c618-2762-41d9-9d4b-360658b47012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:04.694488 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04c618-2762-41d9-9d4b-360658b47012 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:04.694772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04c618-2762-41d9-9d4b-360658b47012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:04.695076 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04c618-2762-41d9-9d4b-360658b47012 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:04.696255 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04c618-2762-41d9-9d4b-360658b47012 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:28:04.706379 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04c618-2762-41d9-9d4b-360658b47012 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:04.706379 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04c618-2762-41d9-9d4b-360658b47012 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:04.714396 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1137/2282] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:28:04.715539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19470d8-6ed6-4d06-a559-867988a6b67a None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:04.725336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19470d8-6ed6-4d06-a559-867988a6b67a None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:04.731680 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce5780-0b12-452e-af70-9b83a8c6d4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:04.732276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce5780-0b12-452e-af70-9b83a8c6d4b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:04.732561 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce5780-0b12-452e-af70-9b83a8c6d4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:04.740891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce5780-0b12-452e-af70-9b83a8c6d4b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:04.740891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce5780-0b12-452e-af70-9b83a8c6d4b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:04.740891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce5780-0b12-452e-af70-9b83a8c6d4b1 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:04.826523 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1146/2283] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:28:04 2022] POST /identity/v3/projects => generated 361 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:28:04.830854 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04c618-2762-41d9-9d4b-360658b47012 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:04.842114 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04c618-2762-41d9-9d4b-360658b47012 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:04.900796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a625bf-1bd4-4ab6-a4f7-9168ccc086ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:04.901541 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a625bf-1bd4-4ab6-a4f7-9168ccc086ce admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:04.901886 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a625bf-1bd4-4ab6-a4f7-9168ccc086ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:04.902218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a625bf-1bd4-4ab6-a4f7-9168ccc086ce admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:04.902902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a625bf-1bd4-4ab6-a4f7-9168ccc086ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:04.905011 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a625bf-1bd4-4ab6-a4f7-9168ccc086ce admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:05.007192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1138/2284] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:28:04 2022] POST /identity/v3/users => generated 281 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:28:05.011946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce5780-0b12-452e-af70-9b83a8c6d4b1 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:05.022302 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce5780-0b12-452e-af70-9b83a8c6d4b1 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:05.027236 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a08e90-d3df-41d0-8547-1d4cf879afbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:05.028014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a08e90-d3df-41d0-8547-1d4cf879afbb admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:05.028294 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a08e90-d3df-41d0-8547-1d4cf879afbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:05.028572 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a08e90-d3df-41d0-8547-1d4cf879afbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:05.029218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a08e90-d3df-41d0-8547-1d4cf879afbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:05.030820 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a08e90-d3df-41d0-8547-1d4cf879afbb admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:05.041481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1147/2285] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:28:05.045413 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a625bf-1bd4-4ab6-a4f7-9168ccc086ce admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:05.056060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a625bf-1bd4-4ab6-a4f7-9168ccc086ce admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:05.060796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ee759-ce62-444f-a1cc-d0550d811c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:05.061393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee759-ce62-444f-a1cc-d0550d811c9d admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:05.061675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee759-ce62-444f-a1cc-d0550d811c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:05.063312 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee759-ce62-444f-a1cc-d0550d811c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:05.063995 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ee759-ce62-444f-a1cc-d0550d811c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:05.065627 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ee759-ce62-444f-a1cc-d0550d811c9d admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:05.077708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1139/2286] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:05.082313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a08e90-d3df-41d0-8547-1d4cf879afbb admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:05.094625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a08e90-d3df-41d0-8547-1d4cf879afbb admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:05.100324 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314197b-9b18-4a96-b395-11e2b42d41d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:05.100984 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314197b-9b18-4a96-b395-11e2b42d41d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:05.101267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314197b-9b18-4a96-b395-11e2b42d41d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:05.101543 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314197b-9b18-4a96-b395-11e2b42d41d2 admin admin] PATH_INFO: `/v3/projects/a7166c7639e242f78e9c2d946df1bd26/users/867c75b4225d440fa1dee1b7c0b28296/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:05.104772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:05.118016 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314197b-9b18-4a96-b395-11e2b42d41d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7166c7639e242f78e9c2d946df1bd26, user_id=867c75b4225d440fa1dee1b7c0b28296, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:05.121017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314197b-9b18-4a96-b395-11e2b42d41d2 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:05.246979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1148/2287] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:05 2022] PUT /identity/v3/projects/a7166c7639e242f78e9c2d946df1bd26/users/867c75b4225d440fa1dee1b7c0b28296/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:05.257284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b399a-6a9f-48e5-ab74-1f4e1210c8bb None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:05.257579 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b399a-6a9f-48e5-ab74-1f4e1210c8bb None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:05.257994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b399a-6a9f-48e5-ab74-1f4e1210c8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:05.288928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-472b399a-6a9f-48e5-ab74-1f4e1210c8bb None None] MFA Rules not processed for user `867c75b4225d440fa1dee1b7c0b28296`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:05.290973 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-472b399a-6a9f-48e5-ab74-1f4e1210c8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:05.364357 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1140/2288] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:28:05.399690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314197b-9b18-4a96-b395-11e2b42d41d2 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:05.401427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7314197b-9b18-4a96-b395-11e2b42d41d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:05.451049 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:05.459844 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314197b-9b18-4a96-b395-11e2b42d41d2 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:05.464427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134da05-a465-4851-9fb0-37475f573dfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:05.465162 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134da05-a465-4851-9fb0-37475f573dfc service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:05.465359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134da05-a465-4851-9fb0-37475f573dfc service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:05.465644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134da05-a465-4851-9fb0-37475f573dfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:05.475685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134da05-a465-4851-9fb0-37475f573dfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:05.477203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134da05-a465-4851-9fb0-37475f573dfc service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:05.487727 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1149/2289] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:05.505072 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b399a-6a9f-48e5-ab74-1f4e1210c8bb None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:05.570161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b399a-6a9f-48e5-ab74-1f4e1210c8bb None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:05.577192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044168a-4d43-4850-a43d-45f861f4306a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:05.578297 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044168a-4d43-4850-a43d-45f861f4306a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:05.578776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044168a-4d43-4850-a43d-45f861f4306a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:05.579266 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044168a-4d43-4850-a43d-45f861f4306a admin admin] PATH_INFO: `/v3/users/867c75b4225d440fa1dee1b7c0b28296` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:05.580920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044168a-4d43-4850-a43d-45f861f4306a admin admin] RBAC: Authorizing `identity:delete_user(user_id=867c75b4225d440fa1dee1b7c0b28296)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:05.582863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044168a-4d43-4850-a43d-45f861f4306a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:05.791183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d044168a-4d43-4850-a43d-45f861f4306a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '867c75b4225d440fa1dee1b7c0b28296'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:05.898035 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d044168a-4d43-4850-a43d-45f861f4306a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '867c75b4225d440fa1dee1b7c0b28296'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:05.899368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d044168a-4d43-4850-a43d-45f861f4306a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '867c75b4225d440fa1dee1b7c0b28296'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:05.913593 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d044168a-4d43-4850-a43d-45f861f4306a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '867c75b4225d440fa1dee1b7c0b28296'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:05.915565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d044168a-4d43-4850-a43d-45f861f4306a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '867c75b4225d440fa1dee1b7c0b28296'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:05.919229 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:05.930684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1141/2290] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:05 2022] DELETE /identity/v3/users/867c75b4225d440fa1dee1b7c0b28296 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:05.935448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e2ed2-6dd0-4cc6-9b4e-d0606f8ba5a4 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:05.935921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e2ed2-6dd0-4cc6-9b4e-d0606f8ba5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:05.936040 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e2ed2-6dd0-4cc6-9b4e-d0606f8ba5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:05.955719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-7e7e2ed2-6dd0-4cc6-9b4e-d0606f8ba5a4 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:05.956945 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7e7e2ed2-6dd0-4cc6-9b4e-d0606f8ba5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:06.021301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1150/2291] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:28:06.032385 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044168a-4d43-4850-a43d-45f861f4306a admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:06.033910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d044168a-4d43-4850-a43d-45f861f4306a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:06.110649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044168a-4d43-4850-a43d-45f861f4306a admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:06.114251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05569e23-5a2a-4d91-93c1-f60730d65bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:06.114864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05569e23-5a2a-4d91-93c1-f60730d65bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:06.115175 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05569e23-5a2a-4d91-93c1-f60730d65bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:06.115455 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05569e23-5a2a-4d91-93c1-f60730d65bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:06.124831 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05569e23-5a2a-4d91-93c1-f60730d65bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:06.126307 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05569e23-5a2a-4d91-93c1-f60730d65bc6 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:06.136498 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1142/2292] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:06.614073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e2ed2-6dd0-4cc6-9b4e-d0606f8ba5a4 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:06.615584 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7e7e2ed2-6dd0-4cc6-9b4e-d0606f8ba5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:06.630044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e2ed2-6dd0-4cc6-9b4e-d0606f8ba5a4 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:06.635682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49438098-31ea-4fbc-b12d-65411e42dc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:06.636325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49438098-31ea-4fbc-b12d-65411e42dc57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:06.636604 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49438098-31ea-4fbc-b12d-65411e42dc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:06.636873 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49438098-31ea-4fbc-b12d-65411e42dc57 admin admin] PATH_INFO: `/v3/projects/a7166c7639e242f78e9c2d946df1bd26` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:06.638006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49438098-31ea-4fbc-b12d-65411e42dc57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7166c7639e242f78e9c2d946df1bd26)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:06.640742 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49438098-31ea-4fbc-b12d-65411e42dc57 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:06.774566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-49438098-31ea-4fbc-b12d-65411e42dc57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7166c7639e242f78e9c2d946df1bd26', 'request_id': 'req-49438098-31ea-4fbc-b12d-65411e42dc57'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:06.780147 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-49438098-31ea-4fbc-b12d-65411e42dc57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7166c7639e242f78e9c2d946df1bd26', 'request_id': 'req-49438098-31ea-4fbc-b12d-65411e42dc57'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:06.840955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-49438098-31ea-4fbc-b12d-65411e42dc57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7166c7639e242f78e9c2d946df1bd26', 'request_id': 'req-49438098-31ea-4fbc-b12d-65411e42dc57'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:06.854199 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-49438098-31ea-4fbc-b12d-65411e42dc57 admin admin] The token cache is being invalidate because project a7166c7639e242f78e9c2d946df1bd26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:06.854516 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-49438098-31ea-4fbc-b12d-65411e42dc57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:06.855200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-49438098-31ea-4fbc-b12d-65411e42dc57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:06.861188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1151/2293] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:06 2022] DELETE /identity/v3/projects/a7166c7639e242f78e9c2d946df1bd26 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:06.862634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f519c7-1484-462c-827f-17f7853b1212 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:06.863271 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f519c7-1484-462c-827f-17f7853b1212 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:06.863365 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f519c7-1484-462c-827f-17f7853b1212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:06.887798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-87f519c7-1484-462c-827f-17f7853b1212 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:06.888915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-87f519c7-1484-462c-827f-17f7853b1212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:06.960325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1143/2294] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:06.968074 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956b43d-0d8b-4235-971f-f986dba9cb1b None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:06.968576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956b43d-0d8b-4235-971f-f986dba9cb1b None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:06.968671 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956b43d-0d8b-4235-971f-f986dba9cb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:06.989235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d956b43d-0d8b-4235-971f-f986dba9cb1b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:06.990445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d956b43d-0d8b-4235-971f-f986dba9cb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:06.999284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1152/2295] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:07.003498 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f519c7-1484-462c-827f-17f7853b1212 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:07.013680 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f519c7-1484-462c-827f-17f7853b1212 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:07.018400 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a42b61-2be1-4bd1-9fbd-a26639379cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:07.019041 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a42b61-2be1-4bd1-9fbd-a26639379cff admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:07.019328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a42b61-2be1-4bd1-9fbd-a26639379cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:07.019605 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a42b61-2be1-4bd1-9fbd-a26639379cff admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:07.020294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a42b61-2be1-4bd1-9fbd-a26639379cff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:28:07.020606 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a42b61-2be1-4bd1-9fbd-a26639379cff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:07.022204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a42b61-2be1-4bd1-9fbd-a26639379cff admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:07.034276 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1144/2296] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:28:07.038725 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956b43d-0d8b-4235-971f-f986dba9cb1b None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:07.044364 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:07.050565 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956b43d-0d8b-4235-971f-f986dba9cb1b None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:07.058000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2121a8-1974-4b88-ab82-412d3ad998c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:07.063230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2121a8-1974-4b88-ab82-412d3ad998c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:07.063230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2121a8-1974-4b88-ab82-412d3ad998c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:07.063230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2121a8-1974-4b88-ab82-412d3ad998c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:07.063230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2121a8-1974-4b88-ab82-412d3ad998c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:07.063230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2121a8-1974-4b88-ab82-412d3ad998c0 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:07.187478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1153/2297] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:28:07 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:07.191455 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a42b61-2be1-4bd1-9fbd-a26639379cff admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:07.194573 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2121a8-1974-4b88-ab82-412d3ad998c0 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:07.196213 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8b2121a8-1974-4b88-ab82-412d3ad998c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:07.201950 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a42b61-2be1-4bd1-9fbd-a26639379cff admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:07.266641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93bad9-f554-4015-82c3-01b63ac37499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:07.267299 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93bad9-f554-4015-82c3-01b63ac37499 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:07.267576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93bad9-f554-4015-82c3-01b63ac37499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:07.267845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93bad9-f554-4015-82c3-01b63ac37499 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:07.268473 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c93bad9-f554-4015-82c3-01b63ac37499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:07.269626 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2121a8-1974-4b88-ab82-412d3ad998c0 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:07.270412 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c93bad9-f554-4015-82c3-01b63ac37499 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:07.280291 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1c59d-8a2d-4c25-8215-194a776a2409 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:07.282766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1c59d-8a2d-4c25-8215-194a776a2409 service placement] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:07.282766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1c59d-8a2d-4c25-8215-194a776a2409 service placement] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:07.282766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1c59d-8a2d-4c25-8215-194a776a2409 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:07.288232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-26a1c59d-8a2d-4c25-8215-194a776a2409 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:07.301388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1c59d-8a2d-4c25-8215-194a776a2409 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:07.303016 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1c59d-8a2d-4c25-8215-194a776a2409 service placement] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:07.312871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1154/2298] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:28:07.393954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1145/2299] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:28:07 2022] POST /identity/v3/users => generated 288 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:07.398334 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1c59d-8a2d-4c25-8215-194a776a2409 service placement] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:07.409466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1c59d-8a2d-4c25-8215-194a776a2409 service placement] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:07.414207 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c03e33-30a1-42b0-ae5d-562b11afe1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:07.415586 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c03e33-30a1-42b0-ae5d-562b11afe1ea admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:07.415936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c03e33-30a1-42b0-ae5d-562b11afe1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:07.416305 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c03e33-30a1-42b0-ae5d-562b11afe1ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:07.417110 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c03e33-30a1-42b0-ae5d-562b11afe1ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:07.418722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c03e33-30a1-42b0-ae5d-562b11afe1ea admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:07.429058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1155/2300] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:07.433295 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93bad9-f554-4015-82c3-01b63ac37499 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:07.440239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:07.443716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93bad9-f554-4015-82c3-01b63ac37499 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:07.448866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866f34e-d757-4d4a-bf18-13b01f3f05b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:07.449477 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866f34e-d757-4d4a-bf18-13b01f3f05b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:07.449758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866f34e-d757-4d4a-bf18-13b01f3f05b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:07.450052 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866f34e-d757-4d4a-bf18-13b01f3f05b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:07.450685 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866f34e-d757-4d4a-bf18-13b01f3f05b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:07.452324 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866f34e-d757-4d4a-bf18-13b01f3f05b5 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:07.464190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1146/2301] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:28:07.468426 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c03e33-30a1-42b0-ae5d-562b11afe1ea admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:07.479757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c03e33-30a1-42b0-ae5d-562b11afe1ea admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:07.484470 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64203b0-2389-467f-b31c-558dc2df54cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:07.485133 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64203b0-2389-467f-b31c-558dc2df54cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:07.485417 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64203b0-2389-467f-b31c-558dc2df54cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:07.485692 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64203b0-2389-467f-b31c-558dc2df54cc admin admin] PATH_INFO: `/v3/projects/349471f366ae46afad3396d4c60be618/users/d461458fbcdf4299b89a493182fcea75/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:07.501227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64203b0-2389-467f-b31c-558dc2df54cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=349471f366ae46afad3396d4c60be618, user_id=d461458fbcdf4299b89a493182fcea75, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:07.503968 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64203b0-2389-467f-b31c-558dc2df54cc admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:07.581915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1156/2302] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:07 2022] PUT /identity/v3/projects/349471f366ae46afad3396d4c60be618/users/d461458fbcdf4299b89a493182fcea75/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:07.593359 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a134e-aa01-4960-a090-2273ab3ab5cb None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:07.593648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a134e-aa01-4960-a090-2273ab3ab5cb None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:07.593924 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a134e-aa01-4960-a090-2273ab3ab5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:07.624744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-8f9a134e-aa01-4960-a090-2273ab3ab5cb None None] MFA Rules not processed for user `d461458fbcdf4299b89a493182fcea75`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:07.625972 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8f9a134e-aa01-4960-a090-2273ab3ab5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:07.696887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1147/2303] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:07.701412 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64203b0-2389-467f-b31c-558dc2df54cc admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:07.768620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64203b0-2389-467f-b31c-558dc2df54cc admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:07.773418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414c7fa-f3a7-4b02-aed9-7cd08e54266a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:07.774758 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414c7fa-f3a7-4b02-aed9-7cd08e54266a admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:07.775078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414c7fa-f3a7-4b02-aed9-7cd08e54266a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:07.775370 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414c7fa-f3a7-4b02-aed9-7cd08e54266a admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:07.776856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414c7fa-f3a7-4b02-aed9-7cd08e54266a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:07.779617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414c7fa-f3a7-4b02-aed9-7cd08e54266a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:07.884901 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1157/2304] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:28:07 2022] POST /identity/v3/projects => generated 375 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:07.885989 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a134e-aa01-4960-a090-2273ab3ab5cb None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:07.897225 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a134e-aa01-4960-a090-2273ab3ab5cb None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:07.959963 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428010f-6fd4-4614-8d02-50a89fd39ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:07.960710 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428010f-6fd4-4614-8d02-50a89fd39ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:07.961067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428010f-6fd4-4614-8d02-50a89fd39ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:07.961910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428010f-6fd4-4614-8d02-50a89fd39ca8 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:07.962544 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428010f-6fd4-4614-8d02-50a89fd39ca8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:07.964539 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428010f-6fd4-4614-8d02-50a89fd39ca8 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:08.070436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1148/2305] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:28:07 2022] POST /identity/v3/users => generated 294 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:28:08.075961 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414c7fa-f3a7-4b02-aed9-7cd08e54266a admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:08.086738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414c7fa-f3a7-4b02-aed9-7cd08e54266a admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:08.091719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0042c5-7eb9-49ea-b6f0-dcf36438d2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:08.092395 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0042c5-7eb9-49ea-b6f0-dcf36438d2da admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:08.092674 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0042c5-7eb9-49ea-b6f0-dcf36438d2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:08.092944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0042c5-7eb9-49ea-b6f0-dcf36438d2da admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:08.093617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0042c5-7eb9-49ea-b6f0-dcf36438d2da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:08.095291 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0042c5-7eb9-49ea-b6f0-dcf36438d2da admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:08.109626 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1158/2306] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:08.122487 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428010f-6fd4-4614-8d02-50a89fd39ca8 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:08.135001 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428010f-6fd4-4614-8d02-50a89fd39ca8 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:08.143403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40a4ed-ca6a-4d0b-8547-8ac820d66564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:08.144069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40a4ed-ca6a-4d0b-8547-8ac820d66564 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:08.144347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40a4ed-ca6a-4d0b-8547-8ac820d66564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:08.144619 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40a4ed-ca6a-4d0b-8547-8ac820d66564 admin admin] PATH_INFO: `/v3/projects/225196a54ea14ba69f8f2658e047e065/users/f6a7e47c3cd7436594fff2535cc7da21/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:08.161363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40a4ed-ca6a-4d0b-8547-8ac820d66564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=225196a54ea14ba69f8f2658e047e065, user_id=f6a7e47c3cd7436594fff2535cc7da21, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:08.164112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40a4ed-ca6a-4d0b-8547-8ac820d66564 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:08.264936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1149/2307] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:08 2022] PUT /identity/v3/projects/225196a54ea14ba69f8f2658e047e065/users/f6a7e47c3cd7436594fff2535cc7da21/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:08.268826 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0042c5-7eb9-49ea-b6f0-dcf36438d2da admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:08.279700 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0042c5-7eb9-49ea-b6f0-dcf36438d2da admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:08.288285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:08.342988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c171b1-117e-45ad-a858-3816a8345d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:08.343659 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c171b1-117e-45ad-a858-3816a8345d7d admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:08.343940 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c171b1-117e-45ad-a858-3816a8345d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:08.344218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c171b1-117e-45ad-a858-3816a8345d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:08.344881 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c171b1-117e-45ad-a858-3816a8345d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:08.346550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c171b1-117e-45ad-a858-3816a8345d7d admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:08.358475 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1159/2308] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:08.363241 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40a4ed-ca6a-4d0b-8547-8ac820d66564 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:08.375307 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40a4ed-ca6a-4d0b-8547-8ac820d66564 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:08.380483 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7099a2-7f37-4432-ad36-51d88733cca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:08.381305 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7099a2-7f37-4432-ad36-51d88733cca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:08.381788 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7099a2-7f37-4432-ad36-51d88733cca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:08.382136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7099a2-7f37-4432-ad36-51d88733cca0 admin admin] PATH_INFO: `/v3/projects/225196a54ea14ba69f8f2658e047e065/users/f6a7e47c3cd7436594fff2535cc7da21/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:08.386420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7099a2-7f37-4432-ad36-51d88733cca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=225196a54ea14ba69f8f2658e047e065, user_id=f6a7e47c3cd7436594fff2535cc7da21, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:08.387725 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7099a2-7f37-4432-ad36-51d88733cca0 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:08.401447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1150/2309] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:08 2022] PUT /identity/v3/projects/225196a54ea14ba69f8f2658e047e065/users/f6a7e47c3cd7436594fff2535cc7da21/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:28:08.412234 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de10f5-35e7-4ff4-bc57-33ddcf70b33c None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:08.412521 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de10f5-35e7-4ff4-bc57-33ddcf70b33c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:08.412793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de10f5-35e7-4ff4-bc57-33ddcf70b33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:08.451144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-89de10f5-35e7-4ff4-bc57-33ddcf70b33c None None] MFA Rules not processed for user `f6a7e47c3cd7436594fff2535cc7da21`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:08.452467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-89de10f5-35e7-4ff4-bc57-33ddcf70b33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:08.538193 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1160/2310] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:08.550826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7099a2-7f37-4432-ad36-51d88733cca0 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:08.552659 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-7b7099a2-7f37-4432-ad36-51d88733cca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:08.603431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:08.622051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7099a2-7f37-4432-ad36-51d88733cca0 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:08.625869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc278dbc-7755-4670-a947-5b21c0aa8fc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:08.626491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc278dbc-7755-4670-a947-5b21c0aa8fc0 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:08.626774 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc278dbc-7755-4670-a947-5b21c0aa8fc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:08.627081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc278dbc-7755-4670-a947-5b21c0aa8fc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:08.687820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc278dbc-7755-4670-a947-5b21c0aa8fc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:08.689310 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc278dbc-7755-4670-a947-5b21c0aa8fc0 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:08.698851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1151/2311] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:13.312533 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de10f5-35e7-4ff4-bc57-33ddcf70b33c None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:13.376799 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de10f5-35e7-4ff4-bc57-33ddcf70b33c None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:13.381281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2fe8b-e2cf-4248-b4d5-fca9c1faccb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:13.381899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2fe8b-e2cf-4248-b4d5-fca9c1faccb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:13.382177 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2fe8b-e2cf-4248-b4d5-fca9c1faccb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:13.382445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2fe8b-e2cf-4248-b4d5-fca9c1faccb4 admin admin] PATH_INFO: `/v3/users/d461458fbcdf4299b89a493182fcea75` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:13.383564 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2fe8b-e2cf-4248-b4d5-fca9c1faccb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d461458fbcdf4299b89a493182fcea75)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:13.385495 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2fe8b-e2cf-4248-b4d5-fca9c1faccb4 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:13.574532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:13.587852 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1fd2fe8b-e2cf-4248-b4d5-fca9c1faccb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd461458fbcdf4299b89a493182fcea75'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:13.601070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1fd2fe8b-e2cf-4248-b4d5-fca9c1faccb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd461458fbcdf4299b89a493182fcea75'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:13.669760 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1fd2fe8b-e2cf-4248-b4d5-fca9c1faccb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd461458fbcdf4299b89a493182fcea75'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:13.677019 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1fd2fe8b-e2cf-4248-b4d5-fca9c1faccb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd461458fbcdf4299b89a493182fcea75'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:13.678978 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1fd2fe8b-e2cf-4248-b4d5-fca9c1faccb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd461458fbcdf4299b89a493182fcea75'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:13.683937 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1161/2312] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:13 2022] DELETE /identity/v3/users/d461458fbcdf4299b89a493182fcea75 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:13.688739 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc15bd5-5b52-4bcf-a3ef-ff7f95e2f82c None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:13.689229 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc15bd5-5b52-4bcf-a3ef-ff7f95e2f82c None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:13.689322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc15bd5-5b52-4bcf-a3ef-ff7f95e2f82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:13.709811 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-3dc15bd5-5b52-4bcf-a3ef-ff7f95e2f82c None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:13.711052 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-3dc15bd5-5b52-4bcf-a3ef-ff7f95e2f82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:13.771479 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1152/2313] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:28:13.782446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2fe8b-e2cf-4248-b4d5-fca9c1faccb4 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:13.783828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-1fd2fe8b-e2cf-4248-b4d5-fca9c1faccb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:13.843754 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2fe8b-e2cf-4248-b4d5-fca9c1faccb4 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:13.847384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedfb5e-3faa-4f36-a701-037c60af82e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:13.848247 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedfb5e-3faa-4f36-a701-037c60af82e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:13.848527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedfb5e-3faa-4f36-a701-037c60af82e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:13.848799 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedfb5e-3faa-4f36-a701-037c60af82e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:13.857950 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fedfb5e-3faa-4f36-a701-037c60af82e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:13.859444 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fedfb5e-3faa-4f36-a701-037c60af82e5 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:13.869301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1162/2314] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:14.293713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc15bd5-5b52-4bcf-a3ef-ff7f95e2f82c None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:14.301074 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-3dc15bd5-5b52-4bcf-a3ef-ff7f95e2f82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:14.307355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc15bd5-5b52-4bcf-a3ef-ff7f95e2f82c None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:14.312279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f5039-78e0-4593-bc9a-391179221768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:14.312918 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f5039-78e0-4593-bc9a-391179221768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:14.313201 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f5039-78e0-4593-bc9a-391179221768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:14.313492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f5039-78e0-4593-bc9a-391179221768 admin admin] PATH_INFO: `/v3/projects/349471f366ae46afad3396d4c60be618` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:14.315528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f5039-78e0-4593-bc9a-391179221768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=349471f366ae46afad3396d4c60be618)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:14.317440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f5039-78e0-4593-bc9a-391179221768 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:14.448254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-b30f5039-78e0-4593-bc9a-391179221768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '349471f366ae46afad3396d4c60be618', 'request_id': 'req-b30f5039-78e0-4593-bc9a-391179221768'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:14.454029 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-b30f5039-78e0-4593-bc9a-391179221768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '349471f366ae46afad3396d4c60be618', 'request_id': 'req-b30f5039-78e0-4593-bc9a-391179221768'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:14.460732 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-b30f5039-78e0-4593-bc9a-391179221768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '349471f366ae46afad3396d4c60be618', 'request_id': 'req-b30f5039-78e0-4593-bc9a-391179221768'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:14.531858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-b30f5039-78e0-4593-bc9a-391179221768 admin admin] The token cache is being invalidate because project 349471f366ae46afad3396d4c60be618 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:14.532136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-b30f5039-78e0-4593-bc9a-391179221768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:14.532772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-b30f5039-78e0-4593-bc9a-391179221768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:14.534268 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1153/2315] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:14 2022] DELETE /identity/v3/projects/349471f366ae46afad3396d4c60be618 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:14.538458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedfb5e-3faa-4f36-a701-037c60af82e5 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:14.549016 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedfb5e-3faa-4f36-a701-037c60af82e5 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:14.606493 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad89d2f-85ad-4f78-859d-fdde5aef6f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:14.607166 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad89d2f-85ad-4f78-859d-fdde5aef6f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:14.607445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad89d2f-85ad-4f78-859d-fdde5aef6f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:14.607714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad89d2f-85ad-4f78-859d-fdde5aef6f20 admin admin] PATH_INFO: `/v3/users/f6a7e47c3cd7436594fff2535cc7da21` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:14.608823 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad89d2f-85ad-4f78-859d-fdde5aef6f20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6a7e47c3cd7436594fff2535cc7da21)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:14.610767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad89d2f-85ad-4f78-859d-fdde5aef6f20 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:14.779215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6ad89d2f-85ad-4f78-859d-fdde5aef6f20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6a7e47c3cd7436594fff2535cc7da21'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:14.785187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:14.792968 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6ad89d2f-85ad-4f78-859d-fdde5aef6f20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6a7e47c3cd7436594fff2535cc7da21'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:14.867351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6ad89d2f-85ad-4f78-859d-fdde5aef6f20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6a7e47c3cd7436594fff2535cc7da21'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:14.875060 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6ad89d2f-85ad-4f78-859d-fdde5aef6f20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6a7e47c3cd7436594fff2535cc7da21'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:14.875717 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6ad89d2f-85ad-4f78-859d-fdde5aef6f20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6a7e47c3cd7436594fff2535cc7da21'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:14.881588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1163/2316] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:14 2022] DELETE /identity/v3/users/f6a7e47c3cd7436594fff2535cc7da21 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:15.320339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f5039-78e0-4593-bc9a-391179221768 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:15.323930 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b30f5039-78e0-4593-bc9a-391179221768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:15.339931 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f5039-78e0-4593-bc9a-391179221768 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:15.349608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:15.403064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99859645-a5d2-4632-9bae-1ef15dd7506a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:15.403711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99859645-a5d2-4632-9bae-1ef15dd7506a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:15.405784 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99859645-a5d2-4632-9bae-1ef15dd7506a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:15.405784 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99859645-a5d2-4632-9bae-1ef15dd7506a admin admin] PATH_INFO: `/v3/projects/225196a54ea14ba69f8f2658e047e065` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:15.405784 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99859645-a5d2-4632-9bae-1ef15dd7506a admin admin] RBAC: Authorizing `identity:delete_project(project_id=225196a54ea14ba69f8f2658e047e065)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:15.407436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99859645-a5d2-4632-9bae-1ef15dd7506a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:15.504771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-99859645-a5d2-4632-9bae-1ef15dd7506a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '225196a54ea14ba69f8f2658e047e065', 'request_id': 'req-99859645-a5d2-4632-9bae-1ef15dd7506a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:15.510523 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-99859645-a5d2-4632-9bae-1ef15dd7506a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '225196a54ea14ba69f8f2658e047e065', 'request_id': 'req-99859645-a5d2-4632-9bae-1ef15dd7506a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:15.516723 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-99859645-a5d2-4632-9bae-1ef15dd7506a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '225196a54ea14ba69f8f2658e047e065', 'request_id': 'req-99859645-a5d2-4632-9bae-1ef15dd7506a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:15.586412 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-99859645-a5d2-4632-9bae-1ef15dd7506a admin admin] The token cache is being invalidate because project 225196a54ea14ba69f8f2658e047e065 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:15.586701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-99859645-a5d2-4632-9bae-1ef15dd7506a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:15.588014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-99859645-a5d2-4632-9bae-1ef15dd7506a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:15.589584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1154/2317] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:15 2022] DELETE /identity/v3/projects/225196a54ea14ba69f8f2658e047e065 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:15.594440 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193b355-dfa5-4156-80bf-13b5c4904bde None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:15.594970 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193b355-dfa5-4156-80bf-13b5c4904bde None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:15.595077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193b355-dfa5-4156-80bf-13b5c4904bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:15.616108 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-3193b355-dfa5-4156-80bf-13b5c4904bde None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:15.617549 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3193b355-dfa5-4156-80bf-13b5c4904bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:15.677941 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1164/2318] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:15.686149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f27b5-dfd9-4d19-a9ae-525f06e385c5 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:15.686518 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f27b5-dfd9-4d19-a9ae-525f06e385c5 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:15.686897 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f27b5-dfd9-4d19-a9ae-525f06e385c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:15.708012 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ca6f27b5-dfd9-4d19-a9ae-525f06e385c5 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:15.709285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ca6f27b5-dfd9-4d19-a9ae-525f06e385c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:15.718129 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1155/2319] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:15.722472 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193b355-dfa5-4156-80bf-13b5c4904bde None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:15.733435 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193b355-dfa5-4156-80bf-13b5c4904bde None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:15.738388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a58e5f-bad4-4db6-91f1-0d02cc8c5e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:15.739033 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a58e5f-bad4-4db6-91f1-0d02cc8c5e6e admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:15.739322 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a58e5f-bad4-4db6-91f1-0d02cc8c5e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:15.739616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a58e5f-bad4-4db6-91f1-0d02cc8c5e6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:15.740295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a58e5f-bad4-4db6-91f1-0d02cc8c5e6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:28:15.740600 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a58e5f-bad4-4db6-91f1-0d02cc8c5e6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:15.742204 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a58e5f-bad4-4db6-91f1-0d02cc8c5e6e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:15.754144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1165/2320] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:28:15.758624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f27b5-dfd9-4d19-a9ae-525f06e385c5 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:15.768095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f27b5-dfd9-4d19-a9ae-525f06e385c5 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:15.772741 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3e5eb-9af3-4075-b2e4-20f2f8463b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:15.773335 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3e5eb-9af3-4075-b2e4-20f2f8463b70 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:15.773616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3e5eb-9af3-4075-b2e4-20f2f8463b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:15.773886 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3e5eb-9af3-4075-b2e4-20f2f8463b70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:15.775060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3e5eb-9af3-4075-b2e4-20f2f8463b70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:15.776939 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3e5eb-9af3-4075-b2e4-20f2f8463b70 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:15.854762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1156/2321] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:28:15 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:15.858936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a58e5f-bad4-4db6-91f1-0d02cc8c5e6e admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:15.869135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a58e5f-bad4-4db6-91f1-0d02cc8c5e6e admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:15.929745 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc742476-125f-4469-adbf-fbb801c41ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:15.930278 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc742476-125f-4469-adbf-fbb801c41ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:15.930553 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc742476-125f-4469-adbf-fbb801c41ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:15.930848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc742476-125f-4469-adbf-fbb801c41ee9 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:15.931563 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc742476-125f-4469-adbf-fbb801c41ee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:15.933555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc742476-125f-4469-adbf-fbb801c41ee9 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:16.013898 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1166/2322] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:28:15 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:16.018724 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3e5eb-9af3-4075-b2e4-20f2f8463b70 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:16.030519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3e5eb-9af3-4075-b2e4-20f2f8463b70 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:16.040447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a42e080-5f98-4406-a9ed-b4915e24ca84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:16.041130 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42e080-5f98-4406-a9ed-b4915e24ca84 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:16.044254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42e080-5f98-4406-a9ed-b4915e24ca84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:16.044254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42e080-5f98-4406-a9ed-b4915e24ca84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:16.044254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a42e080-5f98-4406-a9ed-b4915e24ca84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:16.044254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a42e080-5f98-4406-a9ed-b4915e24ca84 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:16.055496 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1157/2323] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:28:16.061947 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc742476-125f-4469-adbf-fbb801c41ee9 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:16.072798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc742476-125f-4469-adbf-fbb801c41ee9 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:16.077713 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d168bb3-d23f-4ad4-8f91-ac6bff993d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:16.078315 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d168bb3-d23f-4ad4-8f91-ac6bff993d62 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:16.078639 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d168bb3-d23f-4ad4-8f91-ac6bff993d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:16.078954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d168bb3-d23f-4ad4-8f91-ac6bff993d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:16.079602 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d168bb3-d23f-4ad4-8f91-ac6bff993d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:16.081217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d168bb3-d23f-4ad4-8f91-ac6bff993d62 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:16.092806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1167/2324] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:16.098526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a42e080-5f98-4406-a9ed-b4915e24ca84 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:16.109607 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a42e080-5f98-4406-a9ed-b4915e24ca84 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:16.114387 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45784734-884e-4aa0-9429-6d8d2c77f985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:16.115086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45784734-884e-4aa0-9429-6d8d2c77f985 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:16.115372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45784734-884e-4aa0-9429-6d8d2c77f985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:16.115652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45784734-884e-4aa0-9429-6d8d2c77f985 admin admin] PATH_INFO: `/v3/projects/92ef41f677d24d69ae37d2c206497d6f/users/c440fa0ce8e6445f8381f0bb74213057/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:16.130685 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45784734-884e-4aa0-9429-6d8d2c77f985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92ef41f677d24d69ae37d2c206497d6f, user_id=c440fa0ce8e6445f8381f0bb74213057, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:16.136213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45784734-884e-4aa0-9429-6d8d2c77f985 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:16.213222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1158/2325] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:16 2022] PUT /identity/v3/projects/92ef41f677d24d69ae37d2c206497d6f/users/c440fa0ce8e6445f8381f0bb74213057/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:16.281076 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fde090-93ed-4454-878e-b8f1884564fe None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:16.281368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fde090-93ed-4454-878e-b8f1884564fe None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:16.281648 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fde090-93ed-4454-878e-b8f1884564fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:16.310842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-79fde090-93ed-4454-878e-b8f1884564fe None None] MFA Rules not processed for user `c440fa0ce8e6445f8381f0bb74213057`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:16.312108 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-79fde090-93ed-4454-878e-b8f1884564fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:16.338757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:16.379177 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1168/2326] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:16.390678 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45784734-884e-4aa0-9429-6d8d2c77f985 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:16.392679 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-45784734-884e-4aa0-9429-6d8d2c77f985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:16.439280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:16.459428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45784734-884e-4aa0-9429-6d8d2c77f985 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:16.468044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb5929-efb6-41ec-be87-4db6d8acee4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:16.468044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb5929-efb6-41ec-be87-4db6d8acee4a service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:16.468044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb5929-efb6-41ec-be87-4db6d8acee4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:16.468044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb5929-efb6-41ec-be87-4db6d8acee4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:16.476547 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb5929-efb6-41ec-be87-4db6d8acee4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:16.478093 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb5929-efb6-41ec-be87-4db6d8acee4a service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:16.499801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1159/2327] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:20.423945 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff1849-818b-4db2-8d34-ba1ca82e8e3e None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:20.424570 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff1849-818b-4db2-8d34-ba1ca82e8e3e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:20.424935 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff1849-818b-4db2-8d34-ba1ca82e8e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:20.448664 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a1ff1849-818b-4db2-8d34-ba1ca82e8e3e None None] MFA Rules not processed for user `c440fa0ce8e6445f8381f0bb74213057`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:20.449947 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a1ff1849-818b-4db2-8d34-ba1ca82e8e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:20.458378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1169/2328] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:21.001915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb5929-efb6-41ec-be87-4db6d8acee4a service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:21.004800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b5fb5929-efb6-41ec-be87-4db6d8acee4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:21.089930 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb5929-efb6-41ec-be87-4db6d8acee4a service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:21.093898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713aa33-9de8-423b-b167-15cd328be5b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:21.094577 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713aa33-9de8-423b-b167-15cd328be5b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:21.095159 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713aa33-9de8-423b-b167-15cd328be5b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:21.095645 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713aa33-9de8-423b-b167-15cd328be5b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:21.098605 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-5713aa33-9de8-423b-b167-15cd328be5b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:21.188183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713aa33-9de8-423b-b167-15cd328be5b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:21.190024 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713aa33-9de8-423b-b167-15cd328be5b7 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:21.202965 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1160/2329] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:30.441815 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=18 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:31.196777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=58 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:31.253079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff1849-818b-4db2-8d34-ba1ca82e8e3e None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:31.254487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a1ff1849-818b-4db2-8d34-ba1ca82e8e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:31.321840 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff1849-818b-4db2-8d34-ba1ca82e8e3e None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:31.327323 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0549ace-be08-4ab3-aa3f-5d53eab8ed99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:31.327951 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0549ace-be08-4ab3-aa3f-5d53eab8ed99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.328227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0549ace-be08-4ab3-aa3f-5d53eab8ed99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.328498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0549ace-be08-4ab3-aa3f-5d53eab8ed99 admin admin] PATH_INFO: `/v3/users/16b7ecd41b754880a769f3e1342bed6b` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:31.329642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0549ace-be08-4ab3-aa3f-5d53eab8ed99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16b7ecd41b754880a769f3e1342bed6b)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:31.331712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0549ace-be08-4ab3-aa3f-5d53eab8ed99 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:31.505158 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e0549ace-be08-4ab3-aa3f-5d53eab8ed99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16b7ecd41b754880a769f3e1342bed6b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:31.520636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e0549ace-be08-4ab3-aa3f-5d53eab8ed99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16b7ecd41b754880a769f3e1342bed6b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:31.588588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e0549ace-be08-4ab3-aa3f-5d53eab8ed99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16b7ecd41b754880a769f3e1342bed6b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:31.596100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e0549ace-be08-4ab3-aa3f-5d53eab8ed99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16b7ecd41b754880a769f3e1342bed6b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:31.596757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e0549ace-be08-4ab3-aa3f-5d53eab8ed99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16b7ecd41b754880a769f3e1342bed6b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:31.603139 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1170/2330] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:31 2022] DELETE /identity/v3/users/16b7ecd41b754880a769f3e1342bed6b => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:31.608289 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216eeef-58f9-4364-bda9-23162c58b6e1 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.608579 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216eeef-58f9-4364-bda9-23162c58b6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.608855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216eeef-58f9-4364-bda9-23162c58b6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:31.628746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-3216eeef-58f9-4364-bda9-23162c58b6e1 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:31.629999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-3216eeef-58f9-4364-bda9-23162c58b6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:31.690116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1161/2331] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:31.699813 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0549ace-be08-4ab3-aa3f-5d53eab8ed99 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:31.701393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e0549ace-be08-4ab3-aa3f-5d53eab8ed99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:31.760517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0549ace-be08-4ab3-aa3f-5d53eab8ed99 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:31.764469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b40e06-61db-42da-8fb4-1914509d44f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:31.765093 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b40e06-61db-42da-8fb4-1914509d44f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.765397 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b40e06-61db-42da-8fb4-1914509d44f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.765674 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b40e06-61db-42da-8fb4-1914509d44f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:31.775247 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b40e06-61db-42da-8fb4-1914509d44f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:31.776689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b40e06-61db-42da-8fb4-1914509d44f8 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:31.786575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1171/2332] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:32.694034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216eeef-58f9-4364-bda9-23162c58b6e1 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:32.696044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-3216eeef-58f9-4364-bda9-23162c58b6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:32.709663 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216eeef-58f9-4364-bda9-23162c58b6e1 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:32.714415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23584aa-57b0-4d28-9401-37adc978dff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:32.715604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23584aa-57b0-4d28-9401-37adc978dff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:32.715877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23584aa-57b0-4d28-9401-37adc978dff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:32.716430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23584aa-57b0-4d28-9401-37adc978dff2 admin admin] PATH_INFO: `/v3/projects/4326d5d73ed341e69cbf9ee14ad380c7` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:32.717541 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23584aa-57b0-4d28-9401-37adc978dff2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4326d5d73ed341e69cbf9ee14ad380c7)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:32.719572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23584aa-57b0-4d28-9401-37adc978dff2 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:32.871446 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a23584aa-57b0-4d28-9401-37adc978dff2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4326d5d73ed341e69cbf9ee14ad380c7', 'request_id': 'req-a23584aa-57b0-4d28-9401-37adc978dff2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:32.877361 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a23584aa-57b0-4d28-9401-37adc978dff2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4326d5d73ed341e69cbf9ee14ad380c7', 'request_id': 'req-a23584aa-57b0-4d28-9401-37adc978dff2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:32.883657 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a23584aa-57b0-4d28-9401-37adc978dff2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4326d5d73ed341e69cbf9ee14ad380c7', 'request_id': 'req-a23584aa-57b0-4d28-9401-37adc978dff2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:32.974928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a23584aa-57b0-4d28-9401-37adc978dff2 admin admin] The token cache is being invalidate because project 4326d5d73ed341e69cbf9ee14ad380c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:32.975143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a23584aa-57b0-4d28-9401-37adc978dff2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:32.975769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a23584aa-57b0-4d28-9401-37adc978dff2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:32.977317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1162/2333] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:32 2022] DELETE /identity/v3/projects/4326d5d73ed341e69cbf9ee14ad380c7 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:32.983043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6c0da-7bc4-49ee-98a7-2de2d1d955e5 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:32.983530 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6c0da-7bc4-49ee-98a7-2de2d1d955e5 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:32.983625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6c0da-7bc4-49ee-98a7-2de2d1d955e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:33.003466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-97b6c0da-7bc4-49ee-98a7-2de2d1d955e5 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:33.004674 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-97b6c0da-7bc4-49ee-98a7-2de2d1d955e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:33.022052 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:33.065575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1172/2334] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:28:33.075097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e396be-0db6-4982-96d7-7aa957927a28 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:33.075387 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e396be-0db6-4982-96d7-7aa957927a28 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:33.075661 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e396be-0db6-4982-96d7-7aa957927a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:33.097375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-25e396be-0db6-4982-96d7-7aa957927a28 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:33.098598 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-25e396be-0db6-4982-96d7-7aa957927a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:33.107025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1163/2335] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:33.111704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b6c0da-7bc4-49ee-98a7-2de2d1d955e5 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:33.121691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b6c0da-7bc4-49ee-98a7-2de2d1d955e5 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:33.126347 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993165f8-9fc6-4d15-a4ba-5791b8270936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:33.126988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993165f8-9fc6-4d15-a4ba-5791b8270936 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:33.127297 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993165f8-9fc6-4d15-a4ba-5791b8270936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:33.127546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993165f8-9fc6-4d15-a4ba-5791b8270936 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:33.128204 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993165f8-9fc6-4d15-a4ba-5791b8270936 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:28:33.128509 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993165f8-9fc6-4d15-a4ba-5791b8270936 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:33.130144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993165f8-9fc6-4d15-a4ba-5791b8270936 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:33.143245 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1173/2336] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:28:33.147747 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e396be-0db6-4982-96d7-7aa957927a28 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:33.158018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e396be-0db6-4982-96d7-7aa957927a28 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:33.162785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049edcf-2f24-4ec0-9d88-9fc5d571ec56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:33.163421 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049edcf-2f24-4ec0-9d88-9fc5d571ec56 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:33.163847 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049edcf-2f24-4ec0-9d88-9fc5d571ec56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:33.164143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049edcf-2f24-4ec0-9d88-9fc5d571ec56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:33.164778 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049edcf-2f24-4ec0-9d88-9fc5d571ec56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:33.166688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049edcf-2f24-4ec0-9d88-9fc5d571ec56 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:33.270691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1164/2337] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:28:33 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:33.274722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993165f8-9fc6-4d15-a4ba-5791b8270936 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:33.285851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993165f8-9fc6-4d15-a4ba-5791b8270936 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:33.341282 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cb96a-4212-4562-a5b3-da5907e0825e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:33.341890 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cb96a-4212-4562-a5b3-da5907e0825e admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:33.342169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cb96a-4212-4562-a5b3-da5907e0825e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:33.342439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cb96a-4212-4562-a5b3-da5907e0825e admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:33.343085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cb96a-4212-4562-a5b3-da5907e0825e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:33.344977 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cb96a-4212-4562-a5b3-da5907e0825e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:33.457580 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1174/2338] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:28:33 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:33.458752 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049edcf-2f24-4ec0-9d88-9fc5d571ec56 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:33.470259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049edcf-2f24-4ec0-9d88-9fc5d571ec56 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:33.475650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee05a5b-1ccd-48c5-bc86-f07f87dcefd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:33.476467 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee05a5b-1ccd-48c5-bc86-f07f87dcefd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:33.476750 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee05a5b-1ccd-48c5-bc86-f07f87dcefd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:33.477022 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee05a5b-1ccd-48c5-bc86-f07f87dcefd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:33.477722 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee05a5b-1ccd-48c5-bc86-f07f87dcefd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:33.479399 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee05a5b-1ccd-48c5-bc86-f07f87dcefd4 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:33.494284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cb96a-4212-4562-a5b3-da5907e0825e admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:33.495315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1165/2339] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:33.505257 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cb96a-4212-4562-a5b3-da5907e0825e admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:33.510207 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e5660-dc89-491c-9f98-03dc5585c620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:33.510820 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e5660-dc89-491c-9f98-03dc5585c620 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:33.511132 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e5660-dc89-491c-9f98-03dc5585c620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:33.511408 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e5660-dc89-491c-9f98-03dc5585c620 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:33.512748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e5660-dc89-491c-9f98-03dc5585c620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:33.514385 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e5660-dc89-491c-9f98-03dc5585c620 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:33.531230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1175/2340] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:33.531726 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee05a5b-1ccd-48c5-bc86-f07f87dcefd4 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:33.544716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee05a5b-1ccd-48c5-bc86-f07f87dcefd4 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:33.551644 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ee5ce-ff31-422d-9f5b-a19de70db935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:33.552485 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ee5ce-ff31-422d-9f5b-a19de70db935 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:33.552773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ee5ce-ff31-422d-9f5b-a19de70db935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:33.553051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ee5ce-ff31-422d-9f5b-a19de70db935 admin admin] PATH_INFO: `/v3/projects/b59a32b67fde4526bbb78d6e0a1cebda/users/ca526289756647429b819f466da8941f/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:33.569891 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ee5ce-ff31-422d-9f5b-a19de70db935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b59a32b67fde4526bbb78d6e0a1cebda, user_id=ca526289756647429b819f466da8941f, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:33.572654 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ee5ce-ff31-422d-9f5b-a19de70db935 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:33.651262 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1166/2341] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:33 2022] PUT /identity/v3/projects/b59a32b67fde4526bbb78d6e0a1cebda/users/ca526289756647429b819f466da8941f/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:33.656755 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeffec3-f8dc-42e6-810e-f81e4a1431e8 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:33.657068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeffec3-f8dc-42e6-810e-f81e4a1431e8 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:33.657370 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeffec3-f8dc-42e6-810e-f81e4a1431e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:33.676807 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-beeffec3-f8dc-42e6-810e-f81e4a1431e8 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:33.678017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-beeffec3-f8dc-42e6-810e-f81e4a1431e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:33.739935 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1176/2342] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:33.750349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ee5ce-ff31-422d-9f5b-a19de70db935 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:33.752102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-2b6ee5ce-ff31-422d-9f5b-a19de70db935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:33.775470 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:33.813204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ee5ce-ff31-422d-9f5b-a19de70db935 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:33.817277 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a78faf-5e4d-4b4c-b999-b843c87b7996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:33.817902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a78faf-5e4d-4b4c-b999-b843c87b7996 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:33.818207 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a78faf-5e4d-4b4c-b999-b843c87b7996 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:33.818482 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a78faf-5e4d-4b4c-b999-b843c87b7996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:33.828521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a78faf-5e4d-4b4c-b999-b843c87b7996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:33.829995 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a78faf-5e4d-4b4c-b999-b843c87b7996 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:33.840402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1167/2343] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:39.253458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632d38f-06ab-46ce-bc90-0fd26b1b4dfc None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:39.253458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632d38f-06ab-46ce-bc90-0fd26b1b4dfc None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:39.254435 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632d38f-06ab-46ce-bc90-0fd26b1b4dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:39.306954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-e632d38f-06ab-46ce-bc90-0fd26b1b4dfc None None] MFA Rules not processed for user `ca526289756647429b819f466da8941f`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:39.308192 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e632d38f-06ab-46ce-bc90-0fd26b1b4dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:39.332876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:39.384577 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1177/2344] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:39.389003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a78faf-5e4d-4b4c-b999-b843c87b7996 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:39.401484 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a78faf-5e4d-4b4c-b999-b843c87b7996 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:39.407021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0478c-8a99-40e9-8310-682af69b7562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:39.407641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0478c-8a99-40e9-8310-682af69b7562 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:39.407924 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0478c-8a99-40e9-8310-682af69b7562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:39.408200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0478c-8a99-40e9-8310-682af69b7562 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:39.408850 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0478c-8a99-40e9-8310-682af69b7562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:39.410764 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0478c-8a99-40e9-8310-682af69b7562 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:39.490924 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1168/2345] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:28:39 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:39.495464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632d38f-06ab-46ce-bc90-0fd26b1b4dfc None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:39.507326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632d38f-06ab-46ce-bc90-0fd26b1b4dfc None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:39.573916 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d264ff-0797-421b-8776-20d9d45fee36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:39.575002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d264ff-0797-421b-8776-20d9d45fee36 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:39.575321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d264ff-0797-421b-8776-20d9d45fee36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:39.575601 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d264ff-0797-421b-8776-20d9d45fee36 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:39.576262 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d264ff-0797-421b-8776-20d9d45fee36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:39.578216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d264ff-0797-421b-8776-20d9d45fee36 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:39.668707 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1178/2346] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:28:39 2022] POST /identity/v3/users => generated 296 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:28:39.672507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0478c-8a99-40e9-8310-682af69b7562 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:39.684216 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0478c-8a99-40e9-8310-682af69b7562 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:39.689201 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c6171-ec48-45cc-a384-bb4310fd3c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:39.689812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c6171-ec48-45cc-a384-bb4310fd3c73 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:39.690101 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c6171-ec48-45cc-a384-bb4310fd3c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:39.690364 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c6171-ec48-45cc-a384-bb4310fd3c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:39.691047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c6171-ec48-45cc-a384-bb4310fd3c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:39.692631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c6171-ec48-45cc-a384-bb4310fd3c73 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:39.704572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1169/2347] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:28:39.713175 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d264ff-0797-421b-8776-20d9d45fee36 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:39.726099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d264ff-0797-421b-8776-20d9d45fee36 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:39.733064 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5c1b9-3dae-43a2-83b1-76744255ec4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:39.733649 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5c1b9-3dae-43a2-83b1-76744255ec4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:39.733649 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5c1b9-3dae-43a2-83b1-76744255ec4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:39.733837 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5c1b9-3dae-43a2-83b1-76744255ec4c admin admin] PATH_INFO: `/v3/projects/40fedc2d723c49eea1de62c97a2b1e75/users/6e03d8ca952e488595e461f1951ea05f/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:39.752885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5c1b9-3dae-43a2-83b1-76744255ec4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=40fedc2d723c49eea1de62c97a2b1e75, user_id=6e03d8ca952e488595e461f1951ea05f, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:39.755604 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5c1b9-3dae-43a2-83b1-76744255ec4c admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:39.858307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1179/2348] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:39 2022] PUT /identity/v3/projects/40fedc2d723c49eea1de62c97a2b1e75/users/6e03d8ca952e488595e461f1951ea05f/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:39.862314 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c6171-ec48-45cc-a384-bb4310fd3c73 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:39.873474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c6171-ec48-45cc-a384-bb4310fd3c73 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:39.935470 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42666d39-e35b-4ae9-b4c9-ab11fd79a4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:39.936092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42666d39-e35b-4ae9-b4c9-ab11fd79a4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:39.936377 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42666d39-e35b-4ae9-b4c9-ab11fd79a4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:39.936652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42666d39-e35b-4ae9-b4c9-ab11fd79a4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:39.937301 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42666d39-e35b-4ae9-b4c9-ab11fd79a4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:39.938958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42666d39-e35b-4ae9-b4c9-ab11fd79a4d9 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:39.949121 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1170/2349] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:39.953390 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5c1b9-3dae-43a2-83b1-76744255ec4c admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:39.964025 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5c1b9-3dae-43a2-83b1-76744255ec4c admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:39.969039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae4fae-8a6f-47fc-bf98-9dcd97cad047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:39.969691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae4fae-8a6f-47fc-bf98-9dcd97cad047 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:39.969976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae4fae-8a6f-47fc-bf98-9dcd97cad047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:39.970255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae4fae-8a6f-47fc-bf98-9dcd97cad047 admin admin] PATH_INFO: `/v3/projects/40fedc2d723c49eea1de62c97a2b1e75/users/6e03d8ca952e488595e461f1951ea05f/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:39.972464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae4fae-8a6f-47fc-bf98-9dcd97cad047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40fedc2d723c49eea1de62c97a2b1e75, user_id=6e03d8ca952e488595e461f1951ea05f, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:39.975161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae4fae-8a6f-47fc-bf98-9dcd97cad047 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:39.987471 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1180/2350] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:39 2022] PUT /identity/v3/projects/40fedc2d723c49eea1de62c97a2b1e75/users/6e03d8ca952e488595e461f1951ea05f/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:45.313470 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d5565-15bd-4f74-be42-49ba202e0693 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:45.314064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d5565-15bd-4f74-be42-49ba202e0693 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:45.314064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d5565-15bd-4f74-be42-49ba202e0693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:45.338179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-c50d5565-15bd-4f74-be42-49ba202e0693 None None] MFA Rules not processed for user `1ee990b1658e49cc8bc2fe15e7dc36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:45.341247 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c50d5565-15bd-4f74-be42-49ba202e0693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:45.413854 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1171/2351] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 31 19:28:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:45.476303 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae4fae-8a6f-47fc-bf98-9dcd97cad047 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:45.478127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-b8ae4fae-8a6f-47fc-bf98-9dcd97cad047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:45.556371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae4fae-8a6f-47fc-bf98-9dcd97cad047 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:45.562974 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f54956-44ac-49f6-989a-80b0a2541e55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25ed58369ffd46e8b7fa1880662926e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:45.563608 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f54956-44ac-49f6-989a-80b0a2541e55 service swift] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:45.563994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f54956-44ac-49f6-989a-80b0a2541e55 service swift] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:45.565585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f54956-44ac-49f6-989a-80b0a2541e55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:45.574541 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f54956-44ac-49f6-989a-80b0a2541e55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:45.577216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f54956-44ac-49f6-989a-80b0a2541e55 service swift] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:45.586339 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:45.592040 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1181/2352] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 31 19:28:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:28:45.967362 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667c870-92ef-4213-8ed4-e8fd50ff94aa None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:45.970052 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667c870-92ef-4213-8ed4-e8fd50ff94aa None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:45.970052 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667c870-92ef-4213-8ed4-e8fd50ff94aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:46.035614 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-e667c870-92ef-4213-8ed4-e8fd50ff94aa None None] MFA Rules not processed for user `6e03d8ca952e488595e461f1951ea05f`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:46.037154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e667c870-92ef-4213-8ed4-e8fd50ff94aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:46.069824 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:46.136379 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1172/2353] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:46.157835 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f54956-44ac-49f6-989a-80b0a2541e55 service swift] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:46.158215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-99f54956-44ac-49f6-989a-80b0a2541e55 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:46.229416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f54956-44ac-49f6-989a-80b0a2541e55 service swift] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:46.234347 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fb289-20d9-4d05-8d2c-cfe6b576d1d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f281d778f1404984b4d778a1c26aef1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:46.234831 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fb289-20d9-4d05-8d2c-cfe6b576d1d0 service glance] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:46.235161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fb289-20d9-4d05-8d2c-cfe6b576d1d0 service glance] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:46.235438 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fb289-20d9-4d05-8d2c-cfe6b576d1d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:46.312777 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3fb289-20d9-4d05-8d2c-cfe6b576d1d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:46.314286 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3fb289-20d9-4d05-8d2c-cfe6b576d1d0 service glance] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:46.327802 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1182/2354] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:46.587050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdb7c9-255c-49ca-ad52-f103ee71bfea None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:46.589714 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdb7c9-255c-49ca-ad52-f103ee71bfea None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:46.589714 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdb7c9-255c-49ca-ad52-f103ee71bfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:46.610303 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-bfcdb7c9-255c-49ca-ad52-f103ee71bfea None None] MFA Rules not processed for user `1ee990b1658e49cc8bc2fe15e7dc36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:46.612077 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-bfcdb7c9-255c-49ca-ad52-f103ee71bfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:46.620457 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1173/2355] 10.222.0.59 () {58 vars in 967 bytes} [Wed Aug 31 19:28:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:46.634241 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fb289-20d9-4d05-8d2c-cfe6b576d1d0 service glance] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:46.659424 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fb289-20d9-4d05-8d2c-cfe6b576d1d0 service glance] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:46.666462 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c8fa9-b441-4319-983a-3c8fb1d0bd4e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25ed58369ffd46e8b7fa1880662926e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:46.667786 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c8fa9-b441-4319-983a-3c8fb1d0bd4e service swift] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:46.668225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c8fa9-b441-4319-983a-3c8fb1d0bd4e service swift] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:46.668655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c8fa9-b441-4319-983a-3c8fb1d0bd4e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:46.671915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d63355-4c21-4a66-9e8c-8c21612ba8a2 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:46.672573 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d63355-4c21-4a66-9e8c-8c21612ba8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:46.673139 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d63355-4c21-4a66-9e8c-8c21612ba8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:46.679275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c8fa9-b441-4319-983a-3c8fb1d0bd4e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:46.680926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c8fa9-b441-4319-983a-3c8fb1d0bd4e service swift] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:46.693253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1183/2356] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 31 19:28:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:28:46.706081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-38d63355-4c21-4a66-9e8c-8c21612ba8a2 None None] MFA Rules not processed for user `1ee990b1658e49cc8bc2fe15e7dc36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:46.710309 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-38d63355-4c21-4a66-9e8c-8c21612ba8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:46.719790 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1174/2357] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 31 19:28:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:46.744225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c8fa9-b441-4319-983a-3c8fb1d0bd4e service swift] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:46.756194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c8fa9-b441-4319-983a-3c8fb1d0bd4e service swift] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:46.763698 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7225d-f0fd-4329-a19a-5eb4b7130ed3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25ed58369ffd46e8b7fa1880662926e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:46.764330 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7225d-f0fd-4329-a19a-5eb4b7130ed3 service swift] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:46.764617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7225d-f0fd-4329-a19a-5eb4b7130ed3 service swift] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:46.764895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7225d-f0fd-4329-a19a-5eb4b7130ed3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:46.774587 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7225d-f0fd-4329-a19a-5eb4b7130ed3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:46.776292 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7225d-f0fd-4329-a19a-5eb4b7130ed3 service swift] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:46.786974 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1184/2358] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 31 19:28:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:28:47.803109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617857f-0839-41f9-bbe8-5d44526b4065 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:47.803699 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617857f-0839-41f9-bbe8-5d44526b4065 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:47.803859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617857f-0839-41f9-bbe8-5d44526b4065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:47.827509 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-a617857f-0839-41f9-bbe8-5d44526b4065 None None] MFA Rules not processed for user `1ee990b1658e49cc8bc2fe15e7dc36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:47.829141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a617857f-0839-41f9-bbe8-5d44526b4065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:47.840955 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1175/2359] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 31 19:28:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:28:47.859979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7225d-f0fd-4329-a19a-5eb4b7130ed3 service swift] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:47.872647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7225d-f0fd-4329-a19a-5eb4b7130ed3 service swift] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:47.878636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c4309-ad7f-41f8-85cf-270e6db80667 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25ed58369ffd46e8b7fa1880662926e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:47.879481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c4309-ad7f-41f8-85cf-270e6db80667 service swift] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:47.879953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c4309-ad7f-41f8-85cf-270e6db80667 service swift] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:47.880229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c4309-ad7f-41f8-85cf-270e6db80667 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:47.898743 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c4309-ad7f-41f8-85cf-270e6db80667 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:47.900396 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c4309-ad7f-41f8-85cf-270e6db80667 service swift] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:47.910460 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1185/2360] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 31 19:28:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:28:48.635326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad711132-ad0f-417f-91e0-11c6d0d4d437 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:48.635618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad711132-ad0f-417f-91e0-11c6d0d4d437 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:48.636702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad711132-ad0f-417f-91e0-11c6d0d4d437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:48.666447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ad711132-ad0f-417f-91e0-11c6d0d4d437 None None] MFA Rules not processed for user `1ee990b1658e49cc8bc2fe15e7dc36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:48.668224 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ad711132-ad0f-417f-91e0-11c6d0d4d437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:48.676835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1176/2361] 10.222.0.59 () {58 vars in 967 bytes} [Wed Aug 31 19:28:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:48.698464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c4309-ad7f-41f8-85cf-270e6db80667 service swift] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:48.712378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c4309-ad7f-41f8-85cf-270e6db80667 service swift] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:48.723170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a9d42-4a85-4ed2-869d-fc9152266c07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25ed58369ffd46e8b7fa1880662926e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:48.724134 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a9d42-4a85-4ed2-869d-fc9152266c07 service swift] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:48.724798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a9d42-4a85-4ed2-869d-fc9152266c07 service swift] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:48.725281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a9d42-4a85-4ed2-869d-fc9152266c07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:48.746959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a9d42-4a85-4ed2-869d-fc9152266c07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:48.749046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a9d42-4a85-4ed2-869d-fc9152266c07 service swift] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:48.771319 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1186/2362] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 31 19:28:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:28:51.957242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad711132-ad0f-417f-91e0-11c6d0d4d437 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:51.958891 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ad711132-ad0f-417f-91e0-11c6d0d4d437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:52.045020 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad711132-ad0f-417f-91e0-11c6d0d4d437 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:52.050001 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08914a0-6920-4e8e-91df-b9e107400da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:52.050722 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08914a0-6920-4e8e-91df-b9e107400da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:52.051204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08914a0-6920-4e8e-91df-b9e107400da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:52.051554 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08914a0-6920-4e8e-91df-b9e107400da6 admin admin] PATH_INFO: `/v3/users/c440fa0ce8e6445f8381f0bb74213057` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:52.052932 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08914a0-6920-4e8e-91df-b9e107400da6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c440fa0ce8e6445f8381f0bb74213057)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:52.054776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08914a0-6920-4e8e-91df-b9e107400da6 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:52.285664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d08914a0-6920-4e8e-91df-b9e107400da6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c440fa0ce8e6445f8381f0bb74213057'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:52.351554 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d08914a0-6920-4e8e-91df-b9e107400da6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c440fa0ce8e6445f8381f0bb74213057'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:52.352343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d08914a0-6920-4e8e-91df-b9e107400da6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c440fa0ce8e6445f8381f0bb74213057'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:52.376634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d08914a0-6920-4e8e-91df-b9e107400da6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c440fa0ce8e6445f8381f0bb74213057'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:52.377379 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-d08914a0-6920-4e8e-91df-b9e107400da6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c440fa0ce8e6445f8381f0bb74213057'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:52.394356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1177/2363] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:51 2022] DELETE /identity/v3/users/c440fa0ce8e6445f8381f0bb74213057 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:52.399030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052afff-96a0-4034-a602-fca6cc9ebb5c None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:52.399548 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052afff-96a0-4034-a602-fca6cc9ebb5c None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:52.399641 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052afff-96a0-4034-a602-fca6cc9ebb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:52.427547 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-5052afff-96a0-4034-a602-fca6cc9ebb5c None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:52.428900 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-5052afff-96a0-4034-a602-fca6cc9ebb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:52.510272 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1187/2364] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:52.521024 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08914a0-6920-4e8e-91df-b9e107400da6 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:52.522854 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d08914a0-6920-4e8e-91df-b9e107400da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:52.547717 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:52.620741 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08914a0-6920-4e8e-91df-b9e107400da6 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:52.626190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28d9dd-e3cb-410a-a453-8ce277458ebe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:52.629162 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28d9dd-e3cb-410a-a453-8ce277458ebe service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:52.632110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28d9dd-e3cb-410a-a453-8ce277458ebe service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:52.632531 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28d9dd-e3cb-410a-a453-8ce277458ebe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:52.678197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28d9dd-e3cb-410a-a453-8ce277458ebe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:52.680091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28d9dd-e3cb-410a-a453-8ce277458ebe service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:52.694521 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1178/2365] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:53.040576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052afff-96a0-4034-a602-fca6cc9ebb5c None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:53.043709 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-5052afff-96a0-4034-a602-fca6cc9ebb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:53.050800 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:53.055970 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052afff-96a0-4034-a602-fca6cc9ebb5c None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:53.060854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cfc3cf-acb4-42e0-9d90-06f07f7087a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:53.061488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfc3cf-acb4-42e0-9d90-06f07f7087a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:53.061773 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfc3cf-acb4-42e0-9d90-06f07f7087a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:53.062048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfc3cf-acb4-42e0-9d90-06f07f7087a5 admin admin] PATH_INFO: `/v3/projects/92ef41f677d24d69ae37d2c206497d6f` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:53.063217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cfc3cf-acb4-42e0-9d90-06f07f7087a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92ef41f677d24d69ae37d2c206497d6f)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:53.065169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cfc3cf-acb4-42e0-9d90-06f07f7087a5 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:53.623454 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-21cfc3cf-acb4-42e0-9d90-06f07f7087a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92ef41f677d24d69ae37d2c206497d6f', 'request_id': 'req-21cfc3cf-acb4-42e0-9d90-06f07f7087a5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:53.629049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-21cfc3cf-acb4-42e0-9d90-06f07f7087a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92ef41f677d24d69ae37d2c206497d6f', 'request_id': 'req-21cfc3cf-acb4-42e0-9d90-06f07f7087a5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:53.719728 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-21cfc3cf-acb4-42e0-9d90-06f07f7087a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92ef41f677d24d69ae37d2c206497d6f', 'request_id': 'req-21cfc3cf-acb4-42e0-9d90-06f07f7087a5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:53.738471 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-21cfc3cf-acb4-42e0-9d90-06f07f7087a5 admin admin] The token cache is being invalidate because project 92ef41f677d24d69ae37d2c206497d6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:53.738891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-21cfc3cf-acb4-42e0-9d90-06f07f7087a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:53.739637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-21cfc3cf-acb4-42e0-9d90-06f07f7087a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:53.741260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1188/2366] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:53 2022] DELETE /identity/v3/projects/92ef41f677d24d69ae37d2c206497d6f => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:57.700038 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cb067-53ed-4234-b257-959f577f9795 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:57.700554 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cb067-53ed-4234-b257-959f577f9795 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:57.701066 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cb067-53ed-4234-b257-959f577f9795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:57.720545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-851cb067-53ed-4234-b257-959f577f9795 None None] MFA Rules not processed for user `ca526289756647429b819f466da8941f`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:57.722130 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-851cb067-53ed-4234-b257-959f577f9795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:57.778995 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1179/2367] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:29:01.823892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d3556-c2c1-4e2b-b1d0-76bdc28332c8 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:29:01.824408 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d3556-c2c1-4e2b-b1d0-76bdc28332c8 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:29:01.824867 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d3556-c2c1-4e2b-b1d0-76bdc28332c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:29:01.857777 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-c74d3556-c2c1-4e2b-b1d0-76bdc28332c8 None None] MFA Rules not processed for user `1ee990b1658e49cc8bc2fe15e7dc36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:29:01.859873 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c74d3556-c2c1-4e2b-b1d0-76bdc28332c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:01.953109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1189/2368] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 31 19:29:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:29:01.968321 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cb067-53ed-4234-b257-959f577f9795 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:29:01.969754 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-851cb067-53ed-4234-b257-959f577f9795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:02.049308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cb067-53ed-4234-b257-959f577f9795 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:29:02.054592 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf9242-f73f-4877-b36f-a0f0a0baec7b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25ed58369ffd46e8b7fa1880662926e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:29:02.055411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf9242-f73f-4877-b36f-a0f0a0baec7b service swift] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:29:02.055543 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf9242-f73f-4877-b36f-a0f0a0baec7b service swift] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:29:02.055801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf9242-f73f-4877-b36f-a0f0a0baec7b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:29:02.065699 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf9242-f73f-4877-b36f-a0f0a0baec7b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:29:02.067332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf9242-f73f-4877-b36f-a0f0a0baec7b service swift] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:29:02.075983 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1180/2369] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 31 19:29:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:29:11.945183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:29:12.072445 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=79 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:29:43.994796 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285274e-b79c-42e6-9468-d41564be1846 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:29:43.995384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285274e-b79c-42e6-9468-d41564be1846 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:29:43.995384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285274e-b79c-42e6-9468-d41564be1846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:29:44.015819 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-1285274e-b79c-42e6-9468-d41564be1846 None None] MFA Rules not processed for user `1ee990b1658e49cc8bc2fe15e7dc36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:29:44.017025 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-1285274e-b79c-42e6-9468-d41564be1846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:44.025092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1190/2370] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 31 19:29:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:29:44.037938 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf9242-f73f-4877-b36f-a0f0a0baec7b service swift] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:29:44.050028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf9242-f73f-4877-b36f-a0f0a0baec7b service swift] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:29:44.055619 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ec819-75ba-4672-bc05-bfd5fb31a3f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25ed58369ffd46e8b7fa1880662926e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:29:44.056223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ec819-75ba-4672-bc05-bfd5fb31a3f5 service swift] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:29:44.056502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ec819-75ba-4672-bc05-bfd5fb31a3f5 service swift] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:29:44.056775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ec819-75ba-4672-bc05-bfd5fb31a3f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:29:44.066125 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ec819-75ba-4672-bc05-bfd5fb31a3f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:29:44.067664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ec819-75ba-4672-bc05-bfd5fb31a3f5 service swift] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:29:44.076088 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1181/2371] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 31 19:29:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:29:45.036719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285274e-b79c-42e6-9468-d41564be1846 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:29:45.038216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-1285274e-b79c-42e6-9468-d41564be1846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:45.107424 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285274e-b79c-42e6-9468-d41564be1846 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:29:45.111668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019e6cb-5f05-4f9e-aa57-0625b6d1b865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:29:45.112650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019e6cb-5f05-4f9e-aa57-0625b6d1b865 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:29:45.113471 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019e6cb-5f05-4f9e-aa57-0625b6d1b865 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:29:45.113471 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019e6cb-5f05-4f9e-aa57-0625b6d1b865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:29:45.114918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a019e6cb-5f05-4f9e-aa57-0625b6d1b865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:45.184155 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019e6cb-5f05-4f9e-aa57-0625b6d1b865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:29:45.185726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019e6cb-5f05-4f9e-aa57-0625b6d1b865 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:29:45.198657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1191/2372] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:29:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:29:54.072604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:29:55.193963 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=53 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:29:55.979393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ec819-75ba-4672-bc05-bfd5fb31a3f5 service swift] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:29:55.985930 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ea7ec819-75ba-4672-bc05-bfd5fb31a3f5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:55.994186 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ec819-75ba-4672-bc05-bfd5fb31a3f5 service swift] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:29:55.997916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e27493-458e-4f44-ab6d-e4e3c8bae02a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:29:55.998510 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e27493-458e-4f44-ab6d-e4e3c8bae02a service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:29:55.998785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e27493-458e-4f44-ab6d-e4e3c8bae02a service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:29:55.999081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e27493-458e-4f44-ab6d-e4e3c8bae02a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:29:56.000951 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d5e27493-458e-4f44-ab6d-e4e3c8bae02a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:56.087424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e27493-458e-4f44-ab6d-e4e3c8bae02a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:29:56.088861 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e27493-458e-4f44-ab6d-e4e3c8bae02a service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:29:56.107064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1182/2373] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:29:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:29:57.298738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019e6cb-5f05-4f9e-aa57-0625b6d1b865 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:29:57.307597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019e6cb-5f05-4f9e-aa57-0625b6d1b865 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:29:57.311169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e32ea-b4ae-481a-93b2-207575f91731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:29:57.311762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e32ea-b4ae-481a-93b2-207575f91731 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:29:57.312037 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e32ea-b4ae-481a-93b2-207575f91731 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:29:57.312308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e32ea-b4ae-481a-93b2-207575f91731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:29:57.313988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-b05e32ea-b4ae-481a-93b2-207575f91731 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:57.328199 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e32ea-b4ae-481a-93b2-207575f91731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:29:57.329599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e32ea-b4ae-481a-93b2-207575f91731 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:29:57.341079 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1192/2374] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:29:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:30:03.871747 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed0e05-d39c-4f34-a4b2-be623e4fd46f None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:30:03.877426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed0e05-d39c-4f34-a4b2-be623e4fd46f None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:30:03.877426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed0e05-d39c-4f34-a4b2-be623e4fd46f None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:30:03.877426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1183/2375] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 31 19:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:30:03.880013 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b7cff-d041-4cbe-b866-a96a316bdbc1 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:30:03.880570 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b7cff-d041-4cbe-b866-a96a316bdbc1 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:30:03.880570 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b7cff-d041-4cbe-b866-a96a316bdbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:30:03.901101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-aa2b7cff-d041-4cbe-b866-a96a316bdbc1 None None] MFA Rules not processed for user `d461422ce98442bcbe6b254a8dad5e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:30:03.902308 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-aa2b7cff-d041-4cbe-b866-a96a316bdbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:30:03.912888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1193/2376] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 31 19:30:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:30:03.925055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed0e05-d39c-4f34-a4b2-be623e4fd46f None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:30:03.928165 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-10ed0e05-d39c-4f34-a4b2-be623e4fd46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:30:03.989538 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed0e05-d39c-4f34-a4b2-be623e4fd46f None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:30:03.993120 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10339a09-092c-4028-8a19-3d8b9c22d8ef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f281d778f1404984b4d778a1c26aef1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:30:03.993724 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10339a09-092c-4028-8a19-3d8b9c22d8ef service glance] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:30:03.994001 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10339a09-092c-4028-8a19-3d8b9c22d8ef service glance] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:30:03.994271 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10339a09-092c-4028-8a19-3d8b9c22d8ef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:30:04.004351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10339a09-092c-4028-8a19-3d8b9c22d8ef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:30:04.005755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10339a09-092c-4028-8a19-3d8b9c22d8ef service glance] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:30:04.016257 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1184/2377] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:30:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:30:04.210890 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc273200-a508-45ad-8f5d-b9c92eacef6e None None] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:30:04.211182 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc273200-a508-45ad-8f5d-b9c92eacef6e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:30:04.211459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc273200-a508-45ad-8f5d-b9c92eacef6e None None] PATH_INFO: `/` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:30:04.212810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1194/2378] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 31 19:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:30:04.217509 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1a9e2-1a41-45a2-ad69-48bd1be54304 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:30:04.217799 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1a9e2-1a41-45a2-ad69-48bd1be54304 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:30:04.218078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1a9e2-1a41-45a2-ad69-48bd1be54304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:30:04.239143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-0ce1a9e2-1a41-45a2-ad69-48bd1be54304 None None] MFA Rules not processed for user `b7a51545146846cc8284b7044b32198a`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:30:04.240357 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-0ce1a9e2-1a41-45a2-ad69-48bd1be54304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:30:04.246944 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1185/2379] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 31 19:30:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:30:04.258653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc273200-a508-45ad-8f5d-b9c92eacef6e None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:30:04.270170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc273200-a508-45ad-8f5d-b9c92eacef6e None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:30:04.273658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccdda2-2e3a-4721-936e-5cc00064211b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:30:04.274251 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccdda2-2e3a-4721-936e-5cc00064211b service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:30:04.274545 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccdda2-2e3a-4721-936e-5cc00064211b service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:30:04.274788 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccdda2-2e3a-4721-936e-5cc00064211b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:30:04.282082 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccdda2-2e3a-4721-936e-5cc00064211b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:30:04.283553 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccdda2-2e3a-4721-936e-5cc00064211b service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:30:04.292117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1195/2380] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:30:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:30:04.375793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f239f07-4b42-4b37-a2ba-fb7a74731cce None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:30:04.376086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f239f07-4b42-4b37-a2ba-fb7a74731cce None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:30:04.376366 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f239f07-4b42-4b37-a2ba-fb7a74731cce None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:30:04.377776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1186/2381] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 31 19:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:30:04.382600 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5563bf-8a1d-4b43-bc7b-3c81065be983 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:30:04.382916 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5563bf-8a1d-4b43-bc7b-3c81065be983 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:30:04.383199 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5563bf-8a1d-4b43-bc7b-3c81065be983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:30:04.403603 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-cf5563bf-8a1d-4b43-bc7b-3c81065be983 None None] MFA Rules not processed for user `d461422ce98442bcbe6b254a8dad5e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:30:04.404809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-cf5563bf-8a1d-4b43-bc7b-3c81065be983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:30:04.414601 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1196/2382] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 31 19:30:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:30:04.425014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f239f07-4b42-4b37-a2ba-fb7a74731cce None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:30:04.436105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f239f07-4b42-4b37-a2ba-fb7a74731cce None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:30:04.439731 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8dbf4-fafd-411e-a6b8-3a804f1e59b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:30:04.440513 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8dbf4-fafd-411e-a6b8-3a804f1e59b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:30:04.440795 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8dbf4-fafd-411e-a6b8-3a804f1e59b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:30:04.441070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8dbf4-fafd-411e-a6b8-3a804f1e59b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:30:04.451381 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8dbf4-fafd-411e-a6b8-3a804f1e59b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:30:04.452844 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8dbf4-fafd-411e-a6b8-3a804f1e59b3 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:30:04.464311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1187/2383] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:30:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:30:09.302583 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb232e-c371-4316-a7ab-aff6ea802a1f None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:30:09.303084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb232e-c371-4316-a7ab-aff6ea802a1f None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:30:09.303188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb232e-c371-4316-a7ab-aff6ea802a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:30:09.322942 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-7dcb232e-c371-4316-a7ab-aff6ea802a1f None None] MFA Rules not processed for user `1ee990b1658e49cc8bc2fe15e7dc36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:30:09.324177 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7dcb232e-c371-4316-a7ab-aff6ea802a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:30:09.331544 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1197/2384] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 31 19:30:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:30:09.344386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8dbf4-fafd-411e-a6b8-3a804f1e59b3 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:30:09.355880 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8dbf4-fafd-411e-a6b8-3a804f1e59b3 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:30:09.360939 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b84e4-c4fe-4fe0-ba7b-e203087cfa29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25ed58369ffd46e8b7fa1880662926e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:30:09.361526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b84e4-c4fe-4fe0-ba7b-e203087cfa29 service swift] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:30:09.361970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b84e4-c4fe-4fe0-ba7b-e203087cfa29 service swift] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:30:09.362116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b84e4-c4fe-4fe0-ba7b-e203087cfa29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:30:09.370184 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b84e4-c4fe-4fe0-ba7b-e203087cfa29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:30:09.371762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b84e4-c4fe-4fe0-ba7b-e203087cfa29 service swift] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:30:09.379673 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1188/2385] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 31 19:30:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 19:30:12.117980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb232e-c371-4316-a7ab-aff6ea802a1f None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:30:12.121897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7dcb232e-c371-4316-a7ab-aff6ea802a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:30:12.138450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb232e-c371-4316-a7ab-aff6ea802a1f None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:30:12.144495 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da87b5-c292-4c74-9628-a4dccb36688f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:30:12.145280 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da87b5-c292-4c74-9628-a4dccb36688f service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:30:12.145595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da87b5-c292-4c74-9628-a4dccb36688f service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:30:12.146456 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da87b5-c292-4c74-9628-a4dccb36688f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:30:12.154470 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e6da87b5-c292-4c74-9628-a4dccb36688f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:30:12.226092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da87b5-c292-4c74-9628-a4dccb36688f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:30:12.227960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da87b5-c292-4c74-9628-a4dccb36688f service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:30:12.237462 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1198/2386] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:30:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:30:19.376491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=75 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:30:22.233069 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=49 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:30:24.414354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b84e4-c4fe-4fe0-ba7b-e203087cfa29 service swift] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:30:24.415944 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4d4b84e4-c4fe-4fe0-ba7b-e203087cfa29 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:30:24.435353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b84e4-c4fe-4fe0-ba7b-e203087cfa29 service swift] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:30:24.442204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef57ac1-5070-4e8d-996a-012963bb151e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:30:24.442815 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef57ac1-5070-4e8d-996a-012963bb151e service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:30:24.443130 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef57ac1-5070-4e8d-996a-012963bb151e service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:30:24.443409 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef57ac1-5070-4e8d-996a-012963bb151e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:30:24.445916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-2ef57ac1-5070-4e8d-996a-012963bb151e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:30:24.512506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef57ac1-5070-4e8d-996a-012963bb151e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:30:24.514571 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef57ac1-5070-4e8d-996a-012963bb151e service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:30:24.526424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1189/2387] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:30:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:30:34.520604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=28 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:30:38.683385 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da87b5-c292-4c74-9628-a4dccb36688f service nova] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:30:38.706434 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da87b5-c292-4c74-9628-a4dccb36688f service nova] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:30:38.706434 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa678481-54ec-446c-8339-810cc836638b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:30:38.706434 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa678481-54ec-446c-8339-810cc836638b service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:30:38.706434 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa678481-54ec-446c-8339-810cc836638b service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:30:38.707520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa678481-54ec-446c-8339-810cc836638b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:30:38.707520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-aa678481-54ec-446c-8339-810cc836638b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:30:38.712602 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa678481-54ec-446c-8339-810cc836638b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:30:38.714207 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa678481-54ec-446c-8339-810cc836638b service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:30:38.724132 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1199/2388] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:30:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:30:43.620535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64771cfe-e18d-4ed7-95fd-72c23f88a1f3 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:30:43.621038 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64771cfe-e18d-4ed7-95fd-72c23f88a1f3 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:30:43.621140 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64771cfe-e18d-4ed7-95fd-72c23f88a1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:30:43.640338 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-64771cfe-e18d-4ed7-95fd-72c23f88a1f3 None None] MFA Rules not processed for user `1ee990b1658e49cc8bc2fe15e7dc36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:30:43.641525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-64771cfe-e18d-4ed7-95fd-72c23f88a1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:30:43.648629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1190/2389] 10.222.0.59 () {58 vars in 967 bytes} [Wed Aug 31 19:30:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:30:43.663155 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa678481-54ec-446c-8339-810cc836638b service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:30:43.675817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa678481-54ec-446c-8339-810cc836638b service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:30:43.681670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db8d9c-0323-48ee-8573-b351be247db2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25ed58369ffd46e8b7fa1880662926e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:30:43.682268 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db8d9c-0323-48ee-8573-b351be247db2 service swift] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:30:43.682554 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db8d9c-0323-48ee-8573-b351be247db2 service swift] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:30:43.682848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db8d9c-0323-48ee-8573-b351be247db2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:30:43.690961 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db8d9c-0323-48ee-8573-b351be247db2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:30:43.692619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db8d9c-0323-48ee-8573-b351be247db2 service swift] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:30:43.700935 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1200/2390] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 31 19:30:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:30:53.634774 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:30:53.697575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=12 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:30:56.471900 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bcf1f-ad94-42e0-9d98-3c1bd8262a75 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:30:56.472454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bcf1f-ad94-42e0-9d98-3c1bd8262a75 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:30:56.472454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bcf1f-ad94-42e0-9d98-3c1bd8262a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:30:56.495647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-0b5bcf1f-ad94-42e0-9d98-3c1bd8262a75 None None] MFA Rules not processed for user `d441bbdb43c5403abf8480146bc15753`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:30:56.496885 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-0b5bcf1f-ad94-42e0-9d98-3c1bd8262a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:30:56.504217 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1191/2391] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:30:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:31:04.162469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db8d9c-0323-48ee-8573-b351be247db2 service swift] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:31:04.178340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db8d9c-0323-48ee-8573-b351be247db2 service swift] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:31:04.184419 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5519608-a5ae-404e-897f-e691a7897c04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:31:04.185041 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5519608-a5ae-404e-897f-e691a7897c04 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:31:04.185347 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5519608-a5ae-404e-897f-e691a7897c04 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:31:04.185620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5519608-a5ae-404e-897f-e691a7897c04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:31:04.187504 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f5519608-a5ae-404e-897f-e691a7897c04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:31:04.200205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5519608-a5ae-404e-897f-e691a7897c04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:31:04.201715 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5519608-a5ae-404e-897f-e691a7897c04 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:31:04.213222 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1201/2392] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:31:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:31:06.486946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:31:14.208695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=6 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:31:20.384992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5bcf1f-ad94-42e0-9d98-3c1bd8262a75 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:31:20.400844 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5bcf1f-ad94-42e0-9d98-3c1bd8262a75 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:31:20.406968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59174407-3bc1-4083-bdcf-3e90cf9b4a70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:31:20.407589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59174407-3bc1-4083-bdcf-3e90cf9b4a70 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:31:20.407875 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59174407-3bc1-4083-bdcf-3e90cf9b4a70 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:31:20.408151 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59174407-3bc1-4083-bdcf-3e90cf9b4a70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:31:20.409959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-59174407-3bc1-4083-bdcf-3e90cf9b4a70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:31:20.423554 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59174407-3bc1-4083-bdcf-3e90cf9b4a70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:31:20.425200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59174407-3bc1-4083-bdcf-3e90cf9b4a70 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:31:20.438461 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1192/2393] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:31:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:31:22.950025 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d6bb5-3468-464d-8017-b7cd6bceea90 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:31:22.950476 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d6bb5-3468-464d-8017-b7cd6bceea90 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:31:22.950576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d6bb5-3468-464d-8017-b7cd6bceea90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:31:22.972297 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-f80d6bb5-3468-464d-8017-b7cd6bceea90 None None] MFA Rules not processed for user `be55f0bb21f044b0a8346cf953b1650e`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:31:22.973637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f80d6bb5-3468-464d-8017-b7cd6bceea90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:31:23.034435 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1202/2394] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:31:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:31:23.355541 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59174407-3bc1-4083-bdcf-3e90cf9b4a70 service nova] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:31:23.365776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59174407-3bc1-4083-bdcf-3e90cf9b4a70 service nova] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:31:23.369539 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487b031-0059-4508-9b8f-350beb06715f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:31:23.370182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487b031-0059-4508-9b8f-350beb06715f service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:31:23.370432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487b031-0059-4508-9b8f-350beb06715f service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:31:23.370711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487b031-0059-4508-9b8f-350beb06715f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:31:23.372460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-2487b031-0059-4508-9b8f-350beb06715f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:31:23.382933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487b031-0059-4508-9b8f-350beb06715f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:31:23.384438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487b031-0059-4508-9b8f-350beb06715f service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:31:23.393084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1193/2395] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:31:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:31:23.795856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d6bb5-3468-464d-8017-b7cd6bceea90 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:31:23.809662 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d6bb5-3468-464d-8017-b7cd6bceea90 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:31:23.816855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962dcc2d-5595-4eec-8efb-e2e2f4e33c21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:31:23.817472 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dcc2d-5595-4eec-8efb-e2e2f4e33c21 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:31:23.817756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dcc2d-5595-4eec-8efb-e2e2f4e33c21 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:31:23.818035 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dcc2d-5595-4eec-8efb-e2e2f4e33c21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:31:23.819832 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-962dcc2d-5595-4eec-8efb-e2e2f4e33c21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:31:23.887725 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962dcc2d-5595-4eec-8efb-e2e2f4e33c21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:31:23.889932 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962dcc2d-5595-4eec-8efb-e2e2f4e33c21 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:31:23.899856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1203/2396] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:31:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:31:25.218303 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487b031-0059-4508-9b8f-350beb06715f service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:31:25.219941 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-2487b031-0059-4508-9b8f-350beb06715f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:31:25.302852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487b031-0059-4508-9b8f-350beb06715f service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:31:25.308270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a319985-99fc-41c6-843f-d63c12588f2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:31:25.308888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a319985-99fc-41c6-843f-d63c12588f2a service placement] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:31:25.309173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a319985-99fc-41c6-843f-d63c12588f2a service placement] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:31:25.309449 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a319985-99fc-41c6-843f-d63c12588f2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:31:25.311501 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-9a319985-99fc-41c6-843f-d63c12588f2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:31:25.324148 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a319985-99fc-41c6-843f-d63c12588f2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:31:25.325800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a319985-99fc-41c6-843f-d63c12588f2a service placement] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:31:25.338832 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1194/2397] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:31:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:31:33.895197 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=53 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:31:35.334172 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=44 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:31:47.091150 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962dcc2d-5595-4eec-8efb-e2e2f4e33c21 service nova] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:31:47.102899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962dcc2d-5595-4eec-8efb-e2e2f4e33c21 service nova] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:31:47.108876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a57c5a-3812-40b8-ab57-916c54523503 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:31:47.113271 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a57c5a-3812-40b8-ab57-916c54523503 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:31:47.113271 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a57c5a-3812-40b8-ab57-916c54523503 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:31:47.113271 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a57c5a-3812-40b8-ab57-916c54523503 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:31:47.113271 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d6a57c5a-3812-40b8-ab57-916c54523503 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:31:47.125305 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a57c5a-3812-40b8-ab57-916c54523503 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:31:47.128271 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a57c5a-3812-40b8-ab57-916c54523503 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:31:47.138179 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1204/2398] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:31:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:31:57.133659 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=6 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:32:14.838608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bfeab-4a42-491c-82a3-6ddadef6ff98 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:14.839496 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bfeab-4a42-491c-82a3-6ddadef6ff98 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:14.839496 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bfeab-4a42-491c-82a3-6ddadef6ff98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:32:14.859684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-d95bfeab-4a42-491c-82a3-6ddadef6ff98 None None] MFA Rules not processed for user `0d5336bcea514f378e06737680ae9bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:32:14.860989 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d95bfeab-4a42-491c-82a3-6ddadef6ff98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:32:14.868867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1195/2399] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:32:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:32:24.854584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:32:48.272261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a57c5a-3812-40b8-ab57-916c54523503 service nova] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:32:48.281944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a57c5a-3812-40b8-ab57-916c54523503 service nova] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:32:48.285958 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce0eef-ca47-4b32-bb82-0d2e6513a00e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:32:48.286769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce0eef-ca47-4b32-bb82-0d2e6513a00e service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:48.287077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce0eef-ca47-4b32-bb82-0d2e6513a00e service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:48.287351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce0eef-ca47-4b32-bb82-0d2e6513a00e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:32:48.289109 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-46ce0eef-ca47-4b32-bb82-0d2e6513a00e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:32:48.355161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce0eef-ca47-4b32-bb82-0d2e6513a00e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:32:48.356587 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce0eef-ca47-4b32-bb82-0d2e6513a00e service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:32:48.366519 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1205/2400] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:32:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:32:51.951860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bfeab-4a42-491c-82a3-6ddadef6ff98 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:32:51.975538 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bfeab-4a42-491c-82a3-6ddadef6ff98 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:32:51.981366 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f409c04-bed1-4eda-98d5-e239dd46d5c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:32:51.982022 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f409c04-bed1-4eda-98d5-e239dd46d5c7 service placement] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:51.982612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f409c04-bed1-4eda-98d5-e239dd46d5c7 service placement] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:51.982901 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f409c04-bed1-4eda-98d5-e239dd46d5c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:32:51.985298 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-0f409c04-bed1-4eda-98d5-e239dd46d5c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:32:52.000691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f409c04-bed1-4eda-98d5-e239dd46d5c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:32:52.002278 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f409c04-bed1-4eda-98d5-e239dd46d5c7 service placement] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:32:52.013977 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1196/2401] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:32:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:32:52.120670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce0eef-ca47-4b32-bb82-0d2e6513a00e service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:32:52.122125 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-46ce0eef-ca47-4b32-bb82-0d2e6513a00e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:32:52.135878 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce0eef-ca47-4b32-bb82-0d2e6513a00e service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:32:52.141100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe095167-fe8a-487f-a07b-54e9678ffa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:32:52.146469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe095167-fe8a-487f-a07b-54e9678ffa45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:52.146469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe095167-fe8a-487f-a07b-54e9678ffa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:52.146469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe095167-fe8a-487f-a07b-54e9678ffa45 admin admin] PATH_INFO: `/v3/users/d441bbdb43c5403abf8480146bc15753` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:32:52.146469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe095167-fe8a-487f-a07b-54e9678ffa45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d441bbdb43c5403abf8480146bc15753)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:32:52.146469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe095167-fe8a-487f-a07b-54e9678ffa45 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:32:52.356283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-fe095167-fe8a-487f-a07b-54e9678ffa45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd441bbdb43c5403abf8480146bc15753'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:32:52.369394 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-fe095167-fe8a-487f-a07b-54e9678ffa45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd441bbdb43c5403abf8480146bc15753'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:32:52.467397 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-fe095167-fe8a-487f-a07b-54e9678ffa45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd441bbdb43c5403abf8480146bc15753'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:32:52.475696 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-fe095167-fe8a-487f-a07b-54e9678ffa45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd441bbdb43c5403abf8480146bc15753'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:32:52.476514 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-fe095167-fe8a-487f-a07b-54e9678ffa45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd441bbdb43c5403abf8480146bc15753'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:32:52.482605 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1206/2402] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:32:52 2022] DELETE /identity/v3/users/d441bbdb43c5403abf8480146bc15753 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:32:52.812982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f409c04-bed1-4eda-98d5-e239dd46d5c7 service placement] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:32:52.826573 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f409c04-bed1-4eda-98d5-e239dd46d5c7 service placement] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:32:52.889245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7eed7-cea9-49d0-a99a-58067a697e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:32:52.889827 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7eed7-cea9-49d0-a99a-58067a697e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:52.890111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7eed7-cea9-49d0-a99a-58067a697e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:52.890493 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7eed7-cea9-49d0-a99a-58067a697e2e admin admin] PATH_INFO: `/v3/projects/a7e6027f7c1241dcb09786ff94df5c2f` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:32:52.891724 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7eed7-cea9-49d0-a99a-58067a697e2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7e6027f7c1241dcb09786ff94df5c2f)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:32:52.893663 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7eed7-cea9-49d0-a99a-58067a697e2e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:32:53.042059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-52e7eed7-cea9-49d0-a99a-58067a697e2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7e6027f7c1241dcb09786ff94df5c2f', 'request_id': 'req-52e7eed7-cea9-49d0-a99a-58067a697e2e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:32:53.047622 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-52e7eed7-cea9-49d0-a99a-58067a697e2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7e6027f7c1241dcb09786ff94df5c2f', 'request_id': 'req-52e7eed7-cea9-49d0-a99a-58067a697e2e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:32:53.054135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-52e7eed7-cea9-49d0-a99a-58067a697e2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7e6027f7c1241dcb09786ff94df5c2f', 'request_id': 'req-52e7eed7-cea9-49d0-a99a-58067a697e2e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:32:53.132203 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-52e7eed7-cea9-49d0-a99a-58067a697e2e admin admin] The token cache is being invalidate because project a7e6027f7c1241dcb09786ff94df5c2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:32:53.132499 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-52e7eed7-cea9-49d0-a99a-58067a697e2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:32:53.133163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-52e7eed7-cea9-49d0-a99a-58067a697e2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:32:53.134690 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1197/2403] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:32:52 2022] DELETE /identity/v3/projects/a7e6027f7c1241dcb09786ff94df5c2f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:32:53.140162 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504ede4-a9fe-412b-af48-a390738691f4 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:53.140948 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504ede4-a9fe-412b-af48-a390738691f4 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:53.141138 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504ede4-a9fe-412b-af48-a390738691f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:32:53.162700 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-c504ede4-a9fe-412b-af48-a390738691f4 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:32:53.163983 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c504ede4-a9fe-412b-af48-a390738691f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:32:53.223554 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1207/2404] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:32:53.231453 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d9ae0-3119-4160-9bda-413504afe9bf None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:53.231740 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d9ae0-3119-4160-9bda-413504afe9bf None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:53.232014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d9ae0-3119-4160-9bda-413504afe9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:32:53.252342 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-b64d9ae0-3119-4160-9bda-413504afe9bf None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:32:53.253538 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b64d9ae0-3119-4160-9bda-413504afe9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:32:53.262159 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1198/2405] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:32:53.266617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504ede4-a9fe-412b-af48-a390738691f4 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:32:53.276651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504ede4-a9fe-412b-af48-a390738691f4 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:32:53.281417 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73256bc5-ef5d-4d73-9574-bd9386026822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:32:53.282042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73256bc5-ef5d-4d73-9574-bd9386026822 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:53.282319 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73256bc5-ef5d-4d73-9574-bd9386026822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:53.282603 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73256bc5-ef5d-4d73-9574-bd9386026822 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:32:53.283345 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73256bc5-ef5d-4d73-9574-bd9386026822 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:32:53.283657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73256bc5-ef5d-4d73-9574-bd9386026822 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:32:53.285276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73256bc5-ef5d-4d73-9574-bd9386026822 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:32:53.287143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:32:53.297940 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1208/2406] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:32:53.311281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d9ae0-3119-4160-9bda-413504afe9bf None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:32:53.321355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d9ae0-3119-4160-9bda-413504afe9bf None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:32:53.326146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f57b44-612f-470f-9130-f2f0747ceda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:32:53.326740 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f57b44-612f-470f-9130-f2f0747ceda0 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:53.327052 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f57b44-612f-470f-9130-f2f0747ceda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:53.327321 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f57b44-612f-470f-9130-f2f0747ceda0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:32:53.327951 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f57b44-612f-470f-9130-f2f0747ceda0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:32:53.330056 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f57b44-612f-470f-9130-f2f0747ceda0 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:32:53.466833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1199/2407] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:32:53 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:32:53.471182 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73256bc5-ef5d-4d73-9574-bd9386026822 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:32:53.481516 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73256bc5-ef5d-4d73-9574-bd9386026822 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:32:53.539547 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8c80b-dec9-4ce4-b1d4-9514b30e6ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:32:53.540164 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8c80b-dec9-4ce4-b1d4-9514b30e6ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:53.540441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8c80b-dec9-4ce4-b1d4-9514b30e6ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:53.540710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8c80b-dec9-4ce4-b1d4-9514b30e6ba0 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:32:53.541380 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc8c80b-dec9-4ce4-b1d4-9514b30e6ba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:32:53.543378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc8c80b-dec9-4ce4-b1d4-9514b30e6ba0 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:32:53.619059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1209/2408] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:32:53 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:32:53.623255 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f57b44-612f-470f-9130-f2f0747ceda0 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:32:53.633963 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f57b44-612f-470f-9130-f2f0747ceda0 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:32:53.639013 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6674af-7c30-40db-a2ec-00bf61e72ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:32:53.643415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6674af-7c30-40db-a2ec-00bf61e72ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:53.643415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6674af-7c30-40db-a2ec-00bf61e72ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:53.643415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6674af-7c30-40db-a2ec-00bf61e72ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:32:53.643415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6674af-7c30-40db-a2ec-00bf61e72ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:32:53.643415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6674af-7c30-40db-a2ec-00bf61e72ce5 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:32:53.653123 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1200/2409] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:32:53.657505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8c80b-dec9-4ce4-b1d4-9514b30e6ba0 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:32:53.668551 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8c80b-dec9-4ce4-b1d4-9514b30e6ba0 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:32:53.673254 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd165198-1989-41ff-967c-0bc8e52dcdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:32:53.673853 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd165198-1989-41ff-967c-0bc8e52dcdae admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:53.674137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd165198-1989-41ff-967c-0bc8e52dcdae admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:53.674463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd165198-1989-41ff-967c-0bc8e52dcdae admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:32:53.675158 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd165198-1989-41ff-967c-0bc8e52dcdae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:32:53.677414 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd165198-1989-41ff-967c-0bc8e52dcdae admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:32:53.688929 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1210/2410] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:32:53.693183 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6674af-7c30-40db-a2ec-00bf61e72ce5 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:32:53.703652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6674af-7c30-40db-a2ec-00bf61e72ce5 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:32:53.708716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14102ae-fce9-46d9-85e5-e30c11d43a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:32:53.709417 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14102ae-fce9-46d9-85e5-e30c11d43a14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:53.709722 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14102ae-fce9-46d9-85e5-e30c11d43a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:53.709995 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14102ae-fce9-46d9-85e5-e30c11d43a14 admin admin] PATH_INFO: `/v3/projects/dc120bdcb47d4b1b872e2132b8ee9f2e/users/991f6d49ad014318bc9c2f5d57b9deb7/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:32:53.726297 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14102ae-fce9-46d9-85e5-e30c11d43a14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc120bdcb47d4b1b872e2132b8ee9f2e, user_id=991f6d49ad014318bc9c2f5d57b9deb7, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:32:53.728917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14102ae-fce9-46d9-85e5-e30c11d43a14 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:32:53.806448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1201/2411] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:32:53 2022] PUT /identity/v3/projects/dc120bdcb47d4b1b872e2132b8ee9f2e/users/991f6d49ad014318bc9c2f5d57b9deb7/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:32:53.811334 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038173d7-eb16-4eb7-a0c5-3628e8d9ba42 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:53.811625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038173d7-eb16-4eb7-a0c5-3628e8d9ba42 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:53.811907 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038173d7-eb16-4eb7-a0c5-3628e8d9ba42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:32:53.837592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-038173d7-eb16-4eb7-a0c5-3628e8d9ba42 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:32:53.839099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-038173d7-eb16-4eb7-a0c5-3628e8d9ba42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:32:53.903154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1211/2412] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:32:53.913818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14102ae-fce9-46d9-85e5-e30c11d43a14 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:32:53.915674 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e14102ae-fce9-46d9-85e5-e30c11d43a14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:32:53.930834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:32:53.975758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14102ae-fce9-46d9-85e5-e30c11d43a14 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:32:53.979602 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82fc34-35b1-43e3-bd57-f937d4a0a121 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:32:53.980158 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82fc34-35b1-43e3-bd57-f937d4a0a121 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:53.980441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82fc34-35b1-43e3-bd57-f937d4a0a121 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:53.980720 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82fc34-35b1-43e3-bd57-f937d4a0a121 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:32:53.990083 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82fc34-35b1-43e3-bd57-f937d4a0a121 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:32:53.991616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82fc34-35b1-43e3-bd57-f937d4a0a121 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:32:54.001757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1202/2413] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:32:59.967007 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9778ea1-1204-4dd6-b2ad-6ab63418adda None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:59.967961 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9778ea1-1204-4dd6-b2ad-6ab63418adda None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:59.968449 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9778ea1-1204-4dd6-b2ad-6ab63418adda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:00.024359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-c9778ea1-1204-4dd6-b2ad-6ab63418adda None None] MFA Rules not processed for user `991f6d49ad014318bc9c2f5d57b9deb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:33:00.025803 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c9778ea1-1204-4dd6-b2ad-6ab63418adda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:00.095542 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:33:00.134477 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1212/2414] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:32:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:33:00.149469 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82fc34-35b1-43e3-bd57-f937d4a0a121 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:00.151285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-7a82fc34-35b1-43e3-bd57-f937d4a0a121 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:00.351746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82fc34-35b1-43e3-bd57-f937d4a0a121 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:00.358438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75d998-a08b-4e4b-8688-ee2bbd665cfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:00.358647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75d998-a08b-4e4b-8688-ee2bbd665cfd service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:00.359046 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75d998-a08b-4e4b-8688-ee2bbd665cfd service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:00.359408 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75d998-a08b-4e4b-8688-ee2bbd665cfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:00.369290 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75d998-a08b-4e4b-8688-ee2bbd665cfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:00.370900 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75d998-a08b-4e4b-8688-ee2bbd665cfd service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:00.385882 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1203/2415] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:33:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:33:01.268979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557bba3-4839-4409-a9bb-12f82e5d6977 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:01.269700 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557bba3-4839-4409-a9bb-12f82e5d6977 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:01.269864 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557bba3-4839-4409-a9bb-12f82e5d6977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:01.299318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-c557bba3-4839-4409-a9bb-12f82e5d6977 None None] MFA Rules not processed for user `8bc4801f1f674f55b57d49e5f09205ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:33:01.301379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c557bba3-4839-4409-a9bb-12f82e5d6977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:01.361845 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1213/2416] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:33:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:33:01.767201 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75d998-a08b-4e4b-8688-ee2bbd665cfd service nova] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:01.779681 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75d998-a08b-4e4b-8688-ee2bbd665cfd service nova] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:01.785768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68956f3c-4da2-4939-8638-a3920e2dc676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:01.786792 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68956f3c-4da2-4939-8638-a3920e2dc676 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:01.787280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68956f3c-4da2-4939-8638-a3920e2dc676 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:01.787562 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68956f3c-4da2-4939-8638-a3920e2dc676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:01.789340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-68956f3c-4da2-4939-8638-a3920e2dc676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:01.802946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68956f3c-4da2-4939-8638-a3920e2dc676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:01.806020 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68956f3c-4da2-4939-8638-a3920e2dc676 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:01.815918 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1204/2417] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:33:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:33:03.454563 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e3e93-d8d8-462f-8326-4912529b6b4f None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:03.455105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e3e93-d8d8-462f-8326-4912529b6b4f None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:03.455313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e3e93-d8d8-462f-8326-4912529b6b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:03.482901 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-0d2e3e93-d8d8-462f-8326-4912529b6b4f None None] MFA Rules not processed for user `991f6d49ad014318bc9c2f5d57b9deb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:33:03.484153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-0d2e3e93-d8d8-462f-8326-4912529b6b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:03.492452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1214/2418] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:33:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:33:06.283463 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a3b38-bcfa-44ee-9639-c1cdff8983be None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:06.283921 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a3b38-bcfa-44ee-9639-c1cdff8983be None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:06.284026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a3b38-bcfa-44ee-9639-c1cdff8983be None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:06.285864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1205/2419] 10.222.0.59 () {54 vars in 961 bytes} [Wed Aug 31 19:33:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:33:06.291215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff99c8-ba48-4cd0-9789-0134d0fb84da None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:06.291825 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff99c8-ba48-4cd0-9789-0134d0fb84da None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:06.291825 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff99c8-ba48-4cd0-9789-0134d0fb84da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:06.368362 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-84ff99c8-ba48-4cd0-9789-0134d0fb84da None None] MFA Rules not processed for user `eced13cc4030423abe63bb1527f6d35a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:33:06.369234 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-84ff99c8-ba48-4cd0-9789-0134d0fb84da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:06.458997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1215/2420] 10.222.0.59 () {58 vars in 1058 bytes} [Wed Aug 31 19:33:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:33:06.466839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a3b38-bcfa-44ee-9639-c1cdff8983be None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:06.482511 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a3b38-bcfa-44ee-9639-c1cdff8983be None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:06.498489 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c631ddf-7745-443f-8b75-e9af7c52e32e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:06.499596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c631ddf-7745-443f-8b75-e9af7c52e32e service placement] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:06.500111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c631ddf-7745-443f-8b75-e9af7c52e32e service placement] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:06.500650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c631ddf-7745-443f-8b75-e9af7c52e32e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:06.503347 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-9c631ddf-7745-443f-8b75-e9af7c52e32e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:06.517832 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c631ddf-7745-443f-8b75-e9af7c52e32e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:06.519812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c631ddf-7745-443f-8b75-e9af7c52e32e service placement] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:06.539441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1206/2421] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:33:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:33:16.455301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=91 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:33:16.527784 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=96 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:33:28.929545 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35b9c5-90a6-4746-abf0-c861f301701a None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:28.930067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35b9c5-90a6-4746-abf0-c861f301701a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:28.930165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35b9c5-90a6-4746-abf0-c861f301701a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:28.950645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-4c35b9c5-90a6-4746-abf0-c861f301701a None None] MFA Rules not processed for user `1ee990b1658e49cc8bc2fe15e7dc36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:33:28.955807 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4c35b9c5-90a6-4746-abf0-c861f301701a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:29.006486 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1216/2422] 10.222.0.59 () {58 vars in 967 bytes} [Wed Aug 31 19:33:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:33:29.146958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c631ddf-7745-443f-8b75-e9af7c52e32e service placement] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:29.148321 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-9c631ddf-7745-443f-8b75-e9af7c52e32e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:29.216829 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c631ddf-7745-443f-8b75-e9af7c52e32e service placement] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:29.222494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196240a3-e468-4f20-a3d7-3f0f78a37b50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25ed58369ffd46e8b7fa1880662926e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:29.223427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196240a3-e468-4f20-a3d7-3f0f78a37b50 service swift] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:29.223657 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196240a3-e468-4f20-a3d7-3f0f78a37b50 service swift] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:29.223765 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196240a3-e468-4f20-a3d7-3f0f78a37b50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:29.232284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196240a3-e468-4f20-a3d7-3f0f78a37b50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:29.233899 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196240a3-e468-4f20-a3d7-3f0f78a37b50 service swift] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:29.242848 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1207/2423] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 31 19:33:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:33:33.167124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35b9c5-90a6-4746-abf0-c861f301701a None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:33.189271 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35b9c5-90a6-4746-abf0-c861f301701a None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:33.194137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe1479-94ce-444f-969a-b5ad9d2263b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:33.194902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe1479-94ce-444f-969a-b5ad9d2263b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:33.195189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe1479-94ce-444f-969a-b5ad9d2263b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:33.195580 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe1479-94ce-444f-969a-b5ad9d2263b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:33.197661 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-befe1479-94ce-444f-969a-b5ad9d2263b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:33.234796 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe1479-94ce-444f-969a-b5ad9d2263b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:33.238443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe1479-94ce-444f-969a-b5ad9d2263b6 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:33.257954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1217/2424] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:33:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:33:37.732861 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed579dc6-0edf-4cc9-8dda-8bb59f3ebfe4 None None] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:37.733698 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed579dc6-0edf-4cc9-8dda-8bb59f3ebfe4 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:37.733985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed579dc6-0edf-4cc9-8dda-8bb59f3ebfe4 None None] PATH_INFO: `/` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:37.735382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1208/2425] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 31 19:33:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:33:37.746366 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153668ec-756e-4cc6-bab1-49bc571d5d58 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:37.746841 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153668ec-756e-4cc6-bab1-49bc571d5d58 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:37.747048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153668ec-756e-4cc6-bab1-49bc571d5d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:37.766532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-153668ec-756e-4cc6-bab1-49bc571d5d58 None None] MFA Rules not processed for user `eced13cc4030423abe63bb1527f6d35a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:33:37.767785 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-153668ec-756e-4cc6-bab1-49bc571d5d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:37.777704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1218/2426] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 31 19:33:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:33:37.792812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed579dc6-0edf-4cc9-8dda-8bb59f3ebfe4 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:37.794585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ed579dc6-0edf-4cc9-8dda-8bb59f3ebfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:37.811867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed579dc6-0edf-4cc9-8dda-8bb59f3ebfe4 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:37.816867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cdbc1-f56f-4737-97b7-94e8f420f8d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:37.817458 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cdbc1-f56f-4737-97b7-94e8f420f8d9 service placement] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:37.817737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cdbc1-f56f-4737-97b7-94e8f420f8d9 service placement] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:37.818006 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cdbc1-f56f-4737-97b7-94e8f420f8d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:37.826655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cdbc1-f56f-4737-97b7-94e8f420f8d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:37.828270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cdbc1-f56f-4737-97b7-94e8f420f8d9 service placement] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:37.838558 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1209/2427] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:33:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:33:37.872839 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153668ec-756e-4cc6-bab1-49bc571d5d58 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:37.874384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-153668ec-756e-4cc6-bab1-49bc571d5d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:37.888519 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153668ec-756e-4cc6-bab1-49bc571d5d58 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:37.893065 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa19f98-1a50-4698-a417-f06039fc3419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:37.893691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa19f98-1a50-4698-a417-f06039fc3419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:37.893967 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa19f98-1a50-4698-a417-f06039fc3419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:37.894236 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa19f98-1a50-4698-a417-f06039fc3419 admin admin] PATH_INFO: `/v3/users/ca526289756647429b819f466da8941f` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:37.895592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa19f98-1a50-4698-a417-f06039fc3419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca526289756647429b819f466da8941f)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:37.897492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa19f98-1a50-4698-a417-f06039fc3419 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:38.064891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1fa19f98-1a50-4698-a417-f06039fc3419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca526289756647429b819f466da8941f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:38.076970 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1fa19f98-1a50-4698-a417-f06039fc3419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca526289756647429b819f466da8941f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:38.151481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1fa19f98-1a50-4698-a417-f06039fc3419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca526289756647429b819f466da8941f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:38.158995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1fa19f98-1a50-4698-a417-f06039fc3419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca526289756647429b819f466da8941f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:38.159745 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-1fa19f98-1a50-4698-a417-f06039fc3419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca526289756647429b819f466da8941f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:38.166495 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1219/2428] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:33:37 2022] DELETE /identity/v3/users/ca526289756647429b819f466da8941f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:33:38.430025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cdbc1-f56f-4737-97b7-94e8f420f8d9 service placement] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:38.441024 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cdbc1-f56f-4737-97b7-94e8f420f8d9 service placement] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:38.500359 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:38.500989 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:38.501269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:38.501543 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c admin admin] PATH_INFO: `/v3/projects/b59a32b67fde4526bbb78d6e0a1cebda` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:38.502661 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b59a32b67fde4526bbb78d6e0a1cebda)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:38.505473 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:38.663904 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b59a32b67fde4526bbb78d6e0a1cebda', 'request_id': 'req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:38.670583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b59a32b67fde4526bbb78d6e0a1cebda', 'request_id': 'req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:38.677312 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b59a32b67fde4526bbb78d6e0a1cebda', 'request_id': 'req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:38.780193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c admin admin] The token cache is being invalidate because project b59a32b67fde4526bbb78d6e0a1cebda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:33:38.780829 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:38.781178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:38.782668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1210/2429] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:33:38 2022] DELETE /identity/v3/projects/b59a32b67fde4526bbb78d6e0a1cebda => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:33:38.786781 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa19f98-1a50-4698-a417-f06039fc3419 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:38.788229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-1fa19f98-1a50-4698-a417-f06039fc3419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:38.801435 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa19f98-1a50-4698-a417-f06039fc3419 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:38.818316 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:33:38.858228 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7e4a5-133d-48eb-9397-616bbfef45f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:38.858851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7e4a5-133d-48eb-9397-616bbfef45f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:38.859162 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7e4a5-133d-48eb-9397-616bbfef45f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:38.859434 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7e4a5-133d-48eb-9397-616bbfef45f9 admin admin] PATH_INFO: `/v3/users/6e03d8ca952e488595e461f1951ea05f` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:38.860540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7e4a5-133d-48eb-9397-616bbfef45f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e03d8ca952e488595e461f1951ea05f)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:38.862546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7e4a5-133d-48eb-9397-616bbfef45f9 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:39.139248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7de7e4a5-133d-48eb-9397-616bbfef45f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e03d8ca952e488595e461f1951ea05f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:39.151760 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7de7e4a5-133d-48eb-9397-616bbfef45f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e03d8ca952e488595e461f1951ea05f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:39.227993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7de7e4a5-133d-48eb-9397-616bbfef45f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e03d8ca952e488595e461f1951ea05f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:39.235634 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7de7e4a5-133d-48eb-9397-616bbfef45f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e03d8ca952e488595e461f1951ea05f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:39.236321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7de7e4a5-133d-48eb-9397-616bbfef45f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e03d8ca952e488595e461f1951ea05f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:39.243733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1220/2430] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:33:38 2022] DELETE /identity/v3/users/6e03d8ca952e488595e461f1951ea05f => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:33:39.481315 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:39.482819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:39.503035 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fc973-7bc0-4cf4-ad4e-22f6e38c0e2c admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:39.504303 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:33:39.559727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e65f4-7053-42e8-81af-e92968b17ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:39.560405 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e65f4-7053-42e8-81af-e92968b17ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:39.560702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e65f4-7053-42e8-81af-e92968b17ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:39.560974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e65f4-7053-42e8-81af-e92968b17ad1 admin admin] PATH_INFO: `/v3/projects/40fedc2d723c49eea1de62c97a2b1e75` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:39.562077 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e65f4-7053-42e8-81af-e92968b17ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40fedc2d723c49eea1de62c97a2b1e75)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:39.564111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e65f4-7053-42e8-81af-e92968b17ad1 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:39.668822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-097e65f4-7053-42e8-81af-e92968b17ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40fedc2d723c49eea1de62c97a2b1e75', 'request_id': 'req-097e65f4-7053-42e8-81af-e92968b17ad1'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:39.674590 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-097e65f4-7053-42e8-81af-e92968b17ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40fedc2d723c49eea1de62c97a2b1e75', 'request_id': 'req-097e65f4-7053-42e8-81af-e92968b17ad1'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:39.681722 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-097e65f4-7053-42e8-81af-e92968b17ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40fedc2d723c49eea1de62c97a2b1e75', 'request_id': 'req-097e65f4-7053-42e8-81af-e92968b17ad1'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:39.758647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-097e65f4-7053-42e8-81af-e92968b17ad1 admin admin] The token cache is being invalidate because project 40fedc2d723c49eea1de62c97a2b1e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:33:39.759211 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-097e65f4-7053-42e8-81af-e92968b17ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:39.759953 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-097e65f4-7053-42e8-81af-e92968b17ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:39.761560 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1211/2431] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:33:39 2022] DELETE /identity/v3/projects/40fedc2d723c49eea1de62c97a2b1e75 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:33:49.239806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=44,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:33:49.765194 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:33:53.147950 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7e4a5-133d-48eb-9397-616bbfef45f9 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:53.149585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7de7e4a5-133d-48eb-9397-616bbfef45f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:53.232586 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7e4a5-133d-48eb-9397-616bbfef45f9 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:53.283052 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f148c22-4bf6-4aa5-8d4f-a7694cbc837c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:53.283677 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f148c22-4bf6-4aa5-8d4f-a7694cbc837c service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:53.283959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f148c22-4bf6-4aa5-8d4f-a7694cbc837c service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:53.284232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f148c22-4bf6-4aa5-8d4f-a7694cbc837c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:53.285973 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8f148c22-4bf6-4aa5-8d4f-a7694cbc837c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:53.348024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f148c22-4bf6-4aa5-8d4f-a7694cbc837c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:53.349494 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f148c22-4bf6-4aa5-8d4f-a7694cbc837c service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:53.359325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1221/2432] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:33:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:34:03.354742 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=77 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:34:07.129377 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84268ae3-37ca-453c-a4c5-2fac5d58c25e None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:34:07.129903 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84268ae3-37ca-453c-a4c5-2fac5d58c25e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:34:07.129903 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84268ae3-37ca-453c-a4c5-2fac5d58c25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:34:07.149239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-84268ae3-37ca-453c-a4c5-2fac5d58c25e None None] MFA Rules not processed for user `be55f0bb21f044b0a8346cf953b1650e`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:34:07.150454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-84268ae3-37ca-453c-a4c5-2fac5d58c25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:34:07.158462 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1212/2433] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:34:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:34:17.143610 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:34:31.839631 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f148c22-4bf6-4aa5-8d4f-a7694cbc837c service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:34:31.841046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8f148c22-4bf6-4aa5-8d4f-a7694cbc837c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:34:31.911798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f148c22-4bf6-4aa5-8d4f-a7694cbc837c service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:34:31.915450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a1ec1-c8b8-4743-b5bd-478bd2505672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:34:31.916059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a1ec1-c8b8-4743-b5bd-478bd2505672 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:34:31.916343 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a1ec1-c8b8-4743-b5bd-478bd2505672 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:34:31.916620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a1ec1-c8b8-4743-b5bd-478bd2505672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:34:31.918348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-952a1ec1-c8b8-4743-b5bd-478bd2505672 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:34:31.981370 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a1ec1-c8b8-4743-b5bd-478bd2505672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:34:31.982798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a1ec1-c8b8-4743-b5bd-478bd2505672 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:34:31.993162 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1222/2434] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:34:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:34:41.988691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=55 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:34:43.130655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f4867-ccce-4b36-8bd8-564949d664fd None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:34:43.131345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f4867-ccce-4b36-8bd8-564949d664fd None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:34:43.131345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f4867-ccce-4b36-8bd8-564949d664fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:34:43.150259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ec2f4867-ccce-4b36-8bd8-564949d664fd None None] MFA Rules not processed for user `1ee990b1658e49cc8bc2fe15e7dc36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:34:43.151535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ec2f4867-ccce-4b36-8bd8-564949d664fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:34:43.213879 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1213/2435] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 31 19:34:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:34:43.339366 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a1ec1-c8b8-4743-b5bd-478bd2505672 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:34:43.340776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-952a1ec1-c8b8-4743-b5bd-478bd2505672 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:34:43.407837 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a1ec1-c8b8-4743-b5bd-478bd2505672 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:34:43.413558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e468e49-bb0a-4a46-b319-3054dcef922b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25ed58369ffd46e8b7fa1880662926e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:34:43.414182 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e468e49-bb0a-4a46-b319-3054dcef922b service swift] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:34:43.414459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e468e49-bb0a-4a46-b319-3054dcef922b service swift] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:34:43.414726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e468e49-bb0a-4a46-b319-3054dcef922b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:34:43.422788 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e468e49-bb0a-4a46-b319-3054dcef922b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:34:43.424417 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e468e49-bb0a-4a46-b319-3054dcef922b service swift] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:34:43.432659 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1223/2436] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 31 19:34:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:34:53.206764 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=25 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:34:53.429557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=32 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:34:58.001309 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f4867-ccce-4b36-8bd8-564949d664fd None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:34:58.002817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ec2f4867-ccce-4b36-8bd8-564949d664fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:34:58.014358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f4867-ccce-4b36-8bd8-564949d664fd None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:34:58.018006 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4af70-b948-441b-8dc0-04cfc662f3e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:34:58.018612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4af70-b948-441b-8dc0-04cfc662f3e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:34:58.018920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4af70-b948-441b-8dc0-04cfc662f3e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:34:58.019196 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4af70-b948-441b-8dc0-04cfc662f3e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:34:58.021053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8cc4af70-b948-441b-8dc0-04cfc662f3e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:34:58.091967 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4af70-b948-441b-8dc0-04cfc662f3e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:34:58.093402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4af70-b948-441b-8dc0-04cfc662f3e4 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:34:58.105261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1214/2437] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:34:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:34:59.292368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e468e49-bb0a-4a46-b319-3054dcef922b service swift] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:34:59.305638 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e468e49-bb0a-4a46-b319-3054dcef922b service swift] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:34:59.307379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d9f82-a352-4431-b8ae-1774d66406c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:34:59.307981 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d9f82-a352-4431-b8ae-1774d66406c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:34:59.308261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d9f82-a352-4431-b8ae-1774d66406c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:34:59.308530 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d9f82-a352-4431-b8ae-1774d66406c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:34:59.310312 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9a0d9f82-a352-4431-b8ae-1774d66406c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:34:59.326924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0d9f82-a352-4431-b8ae-1774d66406c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:34:59.328399 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0d9f82-a352-4431-b8ae-1774d66406c2 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:34:59.341525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1224/2438] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:34:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:35:05.610694 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b16f2-acc5-4dc5-85f1-87fc0d02b777 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:05.616039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b16f2-acc5-4dc5-85f1-87fc0d02b777 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:05.616039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b16f2-acc5-4dc5-85f1-87fc0d02b777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:05.633310 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-e27b16f2-acc5-4dc5-85f1-87fc0d02b777 None None] MFA Rules not processed for user `1ee990b1658e49cc8bc2fe15e7dc36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:05.635320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e27b16f2-acc5-4dc5-85f1-87fc0d02b777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:05.642157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1215/2439] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 31 19:35:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:05.654686 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d9f82-a352-4431-b8ae-1774d66406c2 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:05.667027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d9f82-a352-4431-b8ae-1774d66406c2 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:05.672070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889986e9-ddb1-45c0-945c-d9cc6c57c87f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25ed58369ffd46e8b7fa1880662926e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:05.672668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889986e9-ddb1-45c0-945c-d9cc6c57c87f service swift] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:05.672948 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889986e9-ddb1-45c0-945c-d9cc6c57c87f service swift] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:05.673334 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889986e9-ddb1-45c0-945c-d9cc6c57c87f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:05.682701 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889986e9-ddb1-45c0-945c-d9cc6c57c87f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:05.684343 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889986e9-ddb1-45c0-945c-d9cc6c57c87f service swift] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:05.693851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1225/2440] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 31 19:35:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:35:12.352213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b16f2-acc5-4dc5-85f1-87fc0d02b777 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:12.354130 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e27b16f2-acc5-4dc5-85f1-87fc0d02b777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:12.372026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b16f2-acc5-4dc5-85f1-87fc0d02b777 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:12.377541 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d25403-e37c-4a35-a06d-6deb56e16311 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:12.378152 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d25403-e37c-4a35-a06d-6deb56e16311 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:12.378432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d25403-e37c-4a35-a06d-6deb56e16311 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:12.378760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d25403-e37c-4a35-a06d-6deb56e16311 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:12.381247 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-10d25403-e37c-4a35-a06d-6deb56e16311 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:12.469150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d25403-e37c-4a35-a06d-6deb56e16311 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:12.470801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d25403-e37c-4a35-a06d-6deb56e16311 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:12.494081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1216/2441] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:35:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:35:15.689707 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=12 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:22.487260 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=78 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:32.141582 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c70156-f0f2-4a20-9be4-065212d32c8d None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:32.142091 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c70156-f0f2-4a20-9be4-065212d32c8d None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:32.142193 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c70156-f0f2-4a20-9be4-065212d32c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:32.196734 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-c3c70156-f0f2-4a20-9be4-065212d32c8d None None] MFA Rules not processed for user `1ee990b1658e49cc8bc2fe15e7dc36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:32.198816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c3c70156-f0f2-4a20-9be4-065212d32c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:32.215928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1226/2442] 10.222.0.59 () {58 vars in 967 bytes} [Wed Aug 31 19:35:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:32.229100 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d25403-e37c-4a35-a06d-6deb56e16311 service nova] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:32.251848 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d25403-e37c-4a35-a06d-6deb56e16311 service nova] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:32.258096 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7825b6-426e-42c0-b86d-1a0ac7bdfb1f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25ed58369ffd46e8b7fa1880662926e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:32.258705 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7825b6-426e-42c0-b86d-1a0ac7bdfb1f service swift] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:32.259021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7825b6-426e-42c0-b86d-1a0ac7bdfb1f service swift] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:32.259302 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7825b6-426e-42c0-b86d-1a0ac7bdfb1f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:32.273612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7825b6-426e-42c0-b86d-1a0ac7bdfb1f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:32.275239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7825b6-426e-42c0-b86d-1a0ac7bdfb1f service swift] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:32.284410 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1217/2443] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 31 19:35:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:35:42.213041 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=18 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:42.280418 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=10 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:46.724444 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d66b8-1a9d-49ba-9895-3b93478ab507 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:46.724927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d66b8-1a9d-49ba-9895-3b93478ab507 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:46.725194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d66b8-1a9d-49ba-9895-3b93478ab507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:46.746468 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-4e0d66b8-1a9d-49ba-9895-3b93478ab507 None None] MFA Rules not processed for user `991f6d49ad014318bc9c2f5d57b9deb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:46.747728 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4e0d66b8-1a9d-49ba-9895-3b93478ab507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:46.803104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1227/2444] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:35:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:52.910164 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7825b6-426e-42c0-b86d-1a0ac7bdfb1f service swift] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:52.921127 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7825b6-426e-42c0-b86d-1a0ac7bdfb1f service swift] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:52.925023 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc7d88-d8f2-4e8d-bc4e-8ee526af4478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:52.925630 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc7d88-d8f2-4e8d-bc4e-8ee526af4478 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:52.926133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc7d88-d8f2-4e8d-bc4e-8ee526af4478 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:52.926407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc7d88-d8f2-4e8d-bc4e-8ee526af4478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:52.928204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b8dc7d88-d8f2-4e8d-bc4e-8ee526af4478 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:52.940123 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc7d88-d8f2-4e8d-bc4e-8ee526af4478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:52.941802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc7d88-d8f2-4e8d-bc4e-8ee526af4478 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:52.950969 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1218/2445] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:35:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:35:56.796207 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=25 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:02.948397 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:08.310750 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d66b8-1a9d-49ba-9895-3b93478ab507 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:08.312363 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4e0d66b8-1a9d-49ba-9895-3b93478ab507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:08.330536 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d66b8-1a9d-49ba-9895-3b93478ab507 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:08.336953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac09a2-6204-4a0f-ba47-e0cfd214c2d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:08.338294 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac09a2-6204-4a0f-ba47-e0cfd214c2d2 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:08.338577 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac09a2-6204-4a0f-ba47-e0cfd214c2d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:08.338856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac09a2-6204-4a0f-ba47-e0cfd214c2d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:08.341371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-daac09a2-6204-4a0f-ba47-e0cfd214c2d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:08.354767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac09a2-6204-4a0f-ba47-e0cfd214c2d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:08.356585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac09a2-6204-4a0f-ba47-e0cfd214c2d2 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:08.369199 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1228/2446] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:12.696020 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc7d88-d8f2-4e8d-bc4e-8ee526af4478 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:12.705423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc7d88-d8f2-4e8d-bc4e-8ee526af4478 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:12.709375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff2d92-c603-465b-925c-9376183c1b5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:12.709964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff2d92-c603-465b-925c-9376183c1b5e service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:12.710242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff2d92-c603-465b-925c-9376183c1b5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:12.710510 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff2d92-c603-465b-925c-9376183c1b5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:12.712300 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-1fff2d92-c603-465b-925c-9376183c1b5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:12.777681 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff2d92-c603-465b-925c-9376183c1b5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:12.779048 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff2d92-c603-465b-925c-9376183c1b5e service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:12.791051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1219/2447] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:16.446085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac09a2-6204-4a0f-ba47-e0cfd214c2d2 service nova] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:16.448400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-daac09a2-6204-4a0f-ba47-e0cfd214c2d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:16.465297 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac09a2-6204-4a0f-ba47-e0cfd214c2d2 service nova] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:16.471236 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f0541-fa5a-4acc-8a2f-5d82e1fb9208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:16.471822 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f0541-fa5a-4acc-8a2f-5d82e1fb9208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:16.472198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f0541-fa5a-4acc-8a2f-5d82e1fb9208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:16.472637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f0541-fa5a-4acc-8a2f-5d82e1fb9208 admin admin] PATH_INFO: `/v3/users/8bc4801f1f674f55b57d49e5f09205ea` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:16.488471 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f0541-fa5a-4acc-8a2f-5d82e1fb9208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bc4801f1f674f55b57d49e5f09205ea)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:16.490403 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f0541-fa5a-4acc-8a2f-5d82e1fb9208 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:16.719084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-b21f0541-fa5a-4acc-8a2f-5d82e1fb9208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bc4801f1f674f55b57d49e5f09205ea'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:16.732504 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-b21f0541-fa5a-4acc-8a2f-5d82e1fb9208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bc4801f1f674f55b57d49e5f09205ea'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:16.806416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-b21f0541-fa5a-4acc-8a2f-5d82e1fb9208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bc4801f1f674f55b57d49e5f09205ea'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:16.814229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-b21f0541-fa5a-4acc-8a2f-5d82e1fb9208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bc4801f1f674f55b57d49e5f09205ea'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:16.814967 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-b21f0541-fa5a-4acc-8a2f-5d82e1fb9208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bc4801f1f674f55b57d49e5f09205ea'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:16.822609 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1229/2448] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:36:16 2022] DELETE /identity/v3/users/8bc4801f1f674f55b57d49e5f09205ea => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:17.048137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff2d92-c603-465b-925c-9376183c1b5e service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:17.060815 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff2d92-c603-465b-925c-9376183c1b5e service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:17.122640 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321ce54-dde5-4e58-a752-2c35ed43ef00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:17.123305 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321ce54-dde5-4e58-a752-2c35ed43ef00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:17.123585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321ce54-dde5-4e58-a752-2c35ed43ef00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:17.123858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321ce54-dde5-4e58-a752-2c35ed43ef00 admin admin] PATH_INFO: `/v3/projects/a13c903a376b46508e8cccbcb20b484c` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:17.135510 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321ce54-dde5-4e58-a752-2c35ed43ef00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a13c903a376b46508e8cccbcb20b484c)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:17.137526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321ce54-dde5-4e58-a752-2c35ed43ef00 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:17.230122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-3321ce54-dde5-4e58-a752-2c35ed43ef00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a13c903a376b46508e8cccbcb20b484c', 'request_id': 'req-3321ce54-dde5-4e58-a752-2c35ed43ef00'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:17.239956 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-3321ce54-dde5-4e58-a752-2c35ed43ef00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a13c903a376b46508e8cccbcb20b484c', 'request_id': 'req-3321ce54-dde5-4e58-a752-2c35ed43ef00'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:17.247366 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-3321ce54-dde5-4e58-a752-2c35ed43ef00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a13c903a376b46508e8cccbcb20b484c', 'request_id': 'req-3321ce54-dde5-4e58-a752-2c35ed43ef00'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:17.320617 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-3321ce54-dde5-4e58-a752-2c35ed43ef00 admin admin] The token cache is being invalidate because project a13c903a376b46508e8cccbcb20b484c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:36:17.320906 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-3321ce54-dde5-4e58-a752-2c35ed43ef00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:17.321690 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-3321ce54-dde5-4e58-a752-2c35ed43ef00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:17.323268 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1220/2449] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:36:17 2022] DELETE /identity/v3/projects/a13c903a376b46508e8cccbcb20b484c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:17.328455 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04887628-4714-48a5-bc1c-ffcb29a70f43 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:17.328739 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04887628-4714-48a5-bc1c-ffcb29a70f43 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:17.329014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04887628-4714-48a5-bc1c-ffcb29a70f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:17.380129 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-04887628-4714-48a5-bc1c-ffcb29a70f43 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:17.381390 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-04887628-4714-48a5-bc1c-ffcb29a70f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:17.443369 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1230/2450] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:17.452710 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe58150-8c86-4c4b-ab3a-f7429650765a None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:17.452988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe58150-8c86-4c4b-ab3a-f7429650765a None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:17.453259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe58150-8c86-4c4b-ab3a-f7429650765a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:17.473891 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-5fe58150-8c86-4c4b-ab3a-f7429650765a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:17.475149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-5fe58150-8c86-4c4b-ab3a-f7429650765a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:17.484459 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1221/2451] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:36:17.488883 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04887628-4714-48a5-bc1c-ffcb29a70f43 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:17.499555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04887628-4714-48a5-bc1c-ffcb29a70f43 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:17.504272 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372d7e0-9284-4c12-b3bb-3d02b85b63a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:17.504870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372d7e0-9284-4c12-b3bb-3d02b85b63a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:17.505151 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372d7e0-9284-4c12-b3bb-3d02b85b63a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:17.505424 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372d7e0-9284-4c12-b3bb-3d02b85b63a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:17.506135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372d7e0-9284-4c12-b3bb-3d02b85b63a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:36:17.506381 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372d7e0-9284-4c12-b3bb-3d02b85b63a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:17.508022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372d7e0-9284-4c12-b3bb-3d02b85b63a1 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:17.519700 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1231/2452] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:36:17.524204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe58150-8c86-4c4b-ab3a-f7429650765a None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:17.533503 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe58150-8c86-4c4b-ab3a-f7429650765a None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:17.538075 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37869a-96fd-4c9b-bee4-d874f07ceb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:17.538672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37869a-96fd-4c9b-bee4-d874f07ceb7e admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:17.538982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37869a-96fd-4c9b-bee4-d874f07ceb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:17.539258 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37869a-96fd-4c9b-bee4-d874f07ceb7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:17.540428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37869a-96fd-4c9b-bee4-d874f07ceb7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:17.542328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37869a-96fd-4c9b-bee4-d874f07ceb7e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:17.547561 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:17.664910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1222/2453] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:36:17 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:17.669729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372d7e0-9284-4c12-b3bb-3d02b85b63a1 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:17.680659 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372d7e0-9284-4c12-b3bb-3d02b85b63a1 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:17.689789 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:17.738626 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfac38-7f3e-4683-bccd-474d4de00c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:17.739284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfac38-7f3e-4683-bccd-474d4de00c98 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:17.739567 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfac38-7f3e-4683-bccd-474d4de00c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:17.739885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfac38-7f3e-4683-bccd-474d4de00c98 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:17.740527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfac38-7f3e-4683-bccd-474d4de00c98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:17.742458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfac38-7f3e-4683-bccd-474d4de00c98 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:17.841597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1232/2454] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:36:17 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:17.845873 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37869a-96fd-4c9b-bee4-d874f07ceb7e admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:17.857464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37869a-96fd-4c9b-bee4-d874f07ceb7e admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:17.862221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351274d5-3eec-49d4-b037-4c8d20793d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:17.862822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351274d5-3eec-49d4-b037-4c8d20793d59 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:17.863138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351274d5-3eec-49d4-b037-4c8d20793d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:17.863413 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351274d5-3eec-49d4-b037-4c8d20793d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:17.864051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351274d5-3eec-49d4-b037-4c8d20793d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:17.865723 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351274d5-3eec-49d4-b037-4c8d20793d59 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:17.876526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1223/2455] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:17.881487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfac38-7f3e-4683-bccd-474d4de00c98 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:17.890979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfac38-7f3e-4683-bccd-474d4de00c98 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:17.896257 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82922d7f-f4c7-4a54-bd8e-8aeef35b20a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:17.896859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82922d7f-f4c7-4a54-bd8e-8aeef35b20a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:17.897144 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82922d7f-f4c7-4a54-bd8e-8aeef35b20a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:17.897415 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82922d7f-f4c7-4a54-bd8e-8aeef35b20a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:17.898096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82922d7f-f4c7-4a54-bd8e-8aeef35b20a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:17.900317 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82922d7f-f4c7-4a54-bd8e-8aeef35b20a3 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:17.911257 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1233/2456] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:17.916015 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351274d5-3eec-49d4-b037-4c8d20793d59 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:17.927871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351274d5-3eec-49d4-b037-4c8d20793d59 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:17.933454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa35858-4409-4024-9c16-9ab7c491dcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:17.934107 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa35858-4409-4024-9c16-9ab7c491dcd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:17.934390 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa35858-4409-4024-9c16-9ab7c491dcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:17.934666 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa35858-4409-4024-9c16-9ab7c491dcd1 admin admin] PATH_INFO: `/v3/projects/cb4338c1b84c4704837c65bff511a19a/users/d577fe0fd40b449183714c56c2f71bb2/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:17.950393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa35858-4409-4024-9c16-9ab7c491dcd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb4338c1b84c4704837c65bff511a19a, user_id=d577fe0fd40b449183714c56c2f71bb2, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:17.953113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa35858-4409-4024-9c16-9ab7c491dcd1 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:18.035320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1224/2457] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:36:17 2022] PUT /identity/v3/projects/cb4338c1b84c4704837c65bff511a19a/users/d577fe0fd40b449183714c56c2f71bb2/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:18.047266 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a8ba3-2ebc-4772-ac7b-3d49efac4304 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:18.047558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a8ba3-2ebc-4772-ac7b-3d49efac4304 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:18.047832 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a8ba3-2ebc-4772-ac7b-3d49efac4304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:18.077419 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-7b3a8ba3-2ebc-4772-ac7b-3d49efac4304 None None] MFA Rules not processed for user `d577fe0fd40b449183714c56c2f71bb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:18.078638 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7b3a8ba3-2ebc-4772-ac7b-3d49efac4304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:18.145187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1234/2458] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:18.150386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa35858-4409-4024-9c16-9ab7c491dcd1 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:18.217702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa35858-4409-4024-9c16-9ab7c491dcd1 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:18.222465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72936ed-1061-4580-a288-b017979216a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:18.223099 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72936ed-1061-4580-a288-b017979216a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:18.223379 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72936ed-1061-4580-a288-b017979216a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:18.223650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72936ed-1061-4580-a288-b017979216a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:18.224277 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72936ed-1061-4580-a288-b017979216a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:18.226149 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72936ed-1061-4580-a288-b017979216a5 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:18.310923 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1225/2459] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:36:18 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:36:18.315048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a8ba3-2ebc-4772-ac7b-3d49efac4304 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:18.326607 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a8ba3-2ebc-4772-ac7b-3d49efac4304 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:18.385145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a371be-ff47-42a1-9618-6f70b76dab44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:18.385766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a371be-ff47-42a1-9618-6f70b76dab44 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:18.386097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a371be-ff47-42a1-9618-6f70b76dab44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:18.386369 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a371be-ff47-42a1-9618-6f70b76dab44 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:18.387175 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a371be-ff47-42a1-9618-6f70b76dab44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:18.389108 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a371be-ff47-42a1-9618-6f70b76dab44 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:18.481362 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1235/2460] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:36:18 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:18.485335 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72936ed-1061-4580-a288-b017979216a5 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:18.496707 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72936ed-1061-4580-a288-b017979216a5 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:18.501603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aabd62-cc39-4d10-a8ed-c2be3ab4d786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:18.502336 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aabd62-cc39-4d10-a8ed-c2be3ab4d786 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:18.502615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aabd62-cc39-4d10-a8ed-c2be3ab4d786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:18.502920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aabd62-cc39-4d10-a8ed-c2be3ab4d786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:18.503568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aabd62-cc39-4d10-a8ed-c2be3ab4d786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:18.505167 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aabd62-cc39-4d10-a8ed-c2be3ab4d786 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:18.516984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1226/2461] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:18.521216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a371be-ff47-42a1-9618-6f70b76dab44 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:18.532141 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a371be-ff47-42a1-9618-6f70b76dab44 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:18.536951 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24b9b8-117c-461a-947c-ededc8f3470b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:18.537619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24b9b8-117c-461a-947c-ededc8f3470b admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:18.537905 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24b9b8-117c-461a-947c-ededc8f3470b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:18.538187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24b9b8-117c-461a-947c-ededc8f3470b admin admin] PATH_INFO: `/v3/projects/75b964dfe4ec4b55b18c16a56566322a/users/3f4902bcd9244a039d549dfbadd25b2e/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:18.547172 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:18.553670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24b9b8-117c-461a-947c-ededc8f3470b admin admin] RBAC: Authorizing `identity:create_grant(project_id=75b964dfe4ec4b55b18c16a56566322a, user_id=3f4902bcd9244a039d549dfbadd25b2e, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:18.557539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24b9b8-117c-461a-947c-ededc8f3470b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:18.669225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1236/2462] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:36:18 2022] PUT /identity/v3/projects/75b964dfe4ec4b55b18c16a56566322a/users/3f4902bcd9244a039d549dfbadd25b2e/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:18.673434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aabd62-cc39-4d10-a8ed-c2be3ab4d786 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:18.684494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aabd62-cc39-4d10-a8ed-c2be3ab4d786 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:18.743880 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ab428-1652-40a7-a066-f37e98e26329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:18.745132 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ab428-1652-40a7-a066-f37e98e26329 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:18.745409 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ab428-1652-40a7-a066-f37e98e26329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:18.745677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ab428-1652-40a7-a066-f37e98e26329 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:18.746313 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ab428-1652-40a7-a066-f37e98e26329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:18.747936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ab428-1652-40a7-a066-f37e98e26329 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:18.757955 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1227/2463] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:18.762982 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24b9b8-117c-461a-947c-ededc8f3470b admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:18.773395 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24b9b8-117c-461a-947c-ededc8f3470b admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:18.778774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a8988-4d85-4738-a561-d6ecb773d03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:18.779463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a8988-4d85-4738-a561-d6ecb773d03e admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:18.779753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a8988-4d85-4738-a561-d6ecb773d03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:18.780029 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a8988-4d85-4738-a561-d6ecb773d03e admin admin] PATH_INFO: `/v3/projects/75b964dfe4ec4b55b18c16a56566322a/users/3f4902bcd9244a039d549dfbadd25b2e/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:18.781964 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a8988-4d85-4738-a561-d6ecb773d03e admin admin] RBAC: Authorizing `identity:create_grant(project_id=75b964dfe4ec4b55b18c16a56566322a, user_id=3f4902bcd9244a039d549dfbadd25b2e, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:18.784678 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a8988-4d85-4738-a561-d6ecb773d03e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:18.796244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1237/2464] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:36:18 2022] PUT /identity/v3/projects/75b964dfe4ec4b55b18c16a56566322a/users/3f4902bcd9244a039d549dfbadd25b2e/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:18.807583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e18fc3-39e8-4703-a01b-b7219d541564 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:18.807874 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e18fc3-39e8-4703-a01b-b7219d541564 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:18.808310 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e18fc3-39e8-4703-a01b-b7219d541564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:18.836359 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:18.839708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-a6e18fc3-39e8-4703-a01b-b7219d541564 None None] MFA Rules not processed for user `3f4902bcd9244a039d549dfbadd25b2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:18.840941 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a6e18fc3-39e8-4703-a01b-b7219d541564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:18.909584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1228/2465] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:18.928995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a8988-4d85-4738-a561-d6ecb773d03e admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:18.931485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-af4a8988-4d85-4738-a561-d6ecb773d03e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:18.988925 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a8988-4d85-4738-a561-d6ecb773d03e admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:18.992685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3cff3-b1ee-4472-8bc8-ccd7d97075fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:18.994008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3cff3-b1ee-4472-8bc8-ccd7d97075fe service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:18.994286 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3cff3-b1ee-4472-8bc8-ccd7d97075fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:18.994562 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3cff3-b1ee-4472-8bc8-ccd7d97075fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:19.003899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3cff3-b1ee-4472-8bc8-ccd7d97075fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:19.005319 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3cff3-b1ee-4472-8bc8-ccd7d97075fe service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:19.015492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1238/2466] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:19.938291 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e18fc3-39e8-4703-a01b-b7219d541564 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:20.002478 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e18fc3-39e8-4703-a01b-b7219d541564 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:20.007234 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a60302-cf18-44f9-a067-16d10ae86f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:20.007858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a60302-cf18-44f9-a067-16d10ae86f49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:20.008139 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a60302-cf18-44f9-a067-16d10ae86f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:20.008415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a60302-cf18-44f9-a067-16d10ae86f49 admin admin] PATH_INFO: `/v3/users/d577fe0fd40b449183714c56c2f71bb2` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:20.009836 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a60302-cf18-44f9-a067-16d10ae86f49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d577fe0fd40b449183714c56c2f71bb2)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:20.011795 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a60302-cf18-44f9-a067-16d10ae86f49 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:20.277317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-f4a60302-cf18-44f9-a067-16d10ae86f49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd577fe0fd40b449183714c56c2f71bb2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:20.354589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-f4a60302-cf18-44f9-a067-16d10ae86f49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd577fe0fd40b449183714c56c2f71bb2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:20.355246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-f4a60302-cf18-44f9-a067-16d10ae86f49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd577fe0fd40b449183714c56c2f71bb2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:20.368268 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-f4a60302-cf18-44f9-a067-16d10ae86f49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd577fe0fd40b449183714c56c2f71bb2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:20.368969 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-f4a60302-cf18-44f9-a067-16d10ae86f49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd577fe0fd40b449183714c56c2f71bb2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:20.381845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1229/2467] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:36:19 2022] DELETE /identity/v3/users/d577fe0fd40b449183714c56c2f71bb2 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:20.386981 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d2594-add3-4ff3-b1c4-743c36d43a8e None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:20.387521 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d2594-add3-4ff3-b1c4-743c36d43a8e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:20.387521 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d2594-add3-4ff3-b1c4-743c36d43a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:20.406864 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-8e0d2594-add3-4ff3-b1c4-743c36d43a8e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:20.408085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8e0d2594-add3-4ff3-b1c4-743c36d43a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:20.467253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1239/2468] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:20.477178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a60302-cf18-44f9-a067-16d10ae86f49 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:20.478759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f4a60302-cf18-44f9-a067-16d10ae86f49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:20.500047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:20.539401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a60302-cf18-44f9-a067-16d10ae86f49 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:20.543279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb5a1f-b265-45e1-b1f1-a50daa262530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:20.543878 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb5a1f-b265-45e1-b1f1-a50daa262530 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:20.544162 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb5a1f-b265-45e1-b1f1-a50daa262530 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:20.544432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb5a1f-b265-45e1-b1f1-a50daa262530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:20.553458 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb5a1f-b265-45e1-b1f1-a50daa262530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:20.554895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb5a1f-b265-45e1-b1f1-a50daa262530 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:20.565294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1230/2469] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:21.025529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d2594-add3-4ff3-b1c4-743c36d43a8e None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:21.027505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8e0d2594-add3-4ff3-b1c4-743c36d43a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:21.039883 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d2594-add3-4ff3-b1c4-743c36d43a8e None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:21.045346 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ceea7-5c3d-44f2-a8f2-a64848431320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:21.046014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ceea7-5c3d-44f2-a8f2-a64848431320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:21.046297 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ceea7-5c3d-44f2-a8f2-a64848431320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:21.046577 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ceea7-5c3d-44f2-a8f2-a64848431320 admin admin] PATH_INFO: `/v3/projects/cb4338c1b84c4704837c65bff511a19a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:21.047793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ceea7-5c3d-44f2-a8f2-a64848431320 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb4338c1b84c4704837c65bff511a19a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:21.049739 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ceea7-5c3d-44f2-a8f2-a64848431320 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:21.166961 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3e8ceea7-5c3d-44f2-a8f2-a64848431320 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb4338c1b84c4704837c65bff511a19a', 'request_id': 'req-3e8ceea7-5c3d-44f2-a8f2-a64848431320'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:21.172705 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3e8ceea7-5c3d-44f2-a8f2-a64848431320 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb4338c1b84c4704837c65bff511a19a', 'request_id': 'req-3e8ceea7-5c3d-44f2-a8f2-a64848431320'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:21.280671 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3e8ceea7-5c3d-44f2-a8f2-a64848431320 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb4338c1b84c4704837c65bff511a19a', 'request_id': 'req-3e8ceea7-5c3d-44f2-a8f2-a64848431320'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:21.290029 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:21.294482 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3e8ceea7-5c3d-44f2-a8f2-a64848431320 admin admin] The token cache is being invalidate because project cb4338c1b84c4704837c65bff511a19a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:36:21.294775 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3e8ceea7-5c3d-44f2-a8f2-a64848431320 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:21.295505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3e8ceea7-5c3d-44f2-a8f2-a64848431320 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:21.297009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1240/2470] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:36:21 2022] DELETE /identity/v3/projects/cb4338c1b84c4704837c65bff511a19a => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:21.301353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb5a1f-b265-45e1-b1f1-a50daa262530 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:21.314514 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb5a1f-b265-45e1-b1f1-a50daa262530 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:21.379915 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277b905-73fc-4664-acc3-0ace336c9f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:21.380560 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277b905-73fc-4664-acc3-0ace336c9f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:21.380841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277b905-73fc-4664-acc3-0ace336c9f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:21.381116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277b905-73fc-4664-acc3-0ace336c9f95 admin admin] PATH_INFO: `/v3/users/3f4902bcd9244a039d549dfbadd25b2e` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:21.382332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277b905-73fc-4664-acc3-0ace336c9f95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f4902bcd9244a039d549dfbadd25b2e)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:21.384220 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277b905-73fc-4664-acc3-0ace336c9f95 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:21.570418 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0277b905-73fc-4664-acc3-0ace336c9f95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f4902bcd9244a039d549dfbadd25b2e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:21.628273 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0277b905-73fc-4664-acc3-0ace336c9f95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f4902bcd9244a039d549dfbadd25b2e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:21.628937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0277b905-73fc-4664-acc3-0ace336c9f95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f4902bcd9244a039d549dfbadd25b2e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:21.641927 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0277b905-73fc-4664-acc3-0ace336c9f95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f4902bcd9244a039d549dfbadd25b2e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:21.642592 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0277b905-73fc-4664-acc3-0ace336c9f95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f4902bcd9244a039d549dfbadd25b2e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:21.656056 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1231/2471] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:36:21 2022] DELETE /identity/v3/users/3f4902bcd9244a039d549dfbadd25b2e => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:22.133253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ceea7-5c3d-44f2-a8f2-a64848431320 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:22.134748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3e8ceea7-5c3d-44f2-a8f2-a64848431320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:22.149156 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ceea7-5c3d-44f2-a8f2-a64848431320 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:22.210305 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e0850-43e3-4ed3-88b9-0ee1879cd7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:22.210976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e0850-43e3-4ed3-88b9-0ee1879cd7ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:22.211258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e0850-43e3-4ed3-88b9-0ee1879cd7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:22.211535 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e0850-43e3-4ed3-88b9-0ee1879cd7ae admin admin] PATH_INFO: `/v3/projects/75b964dfe4ec4b55b18c16a56566322a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:22.212671 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e0850-43e3-4ed3-88b9-0ee1879cd7ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=75b964dfe4ec4b55b18c16a56566322a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:22.215646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e0850-43e3-4ed3-88b9-0ee1879cd7ae admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:22.383468 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-839e0850-43e3-4ed3-88b9-0ee1879cd7ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75b964dfe4ec4b55b18c16a56566322a', 'request_id': 'req-839e0850-43e3-4ed3-88b9-0ee1879cd7ae'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:22.390018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-839e0850-43e3-4ed3-88b9-0ee1879cd7ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75b964dfe4ec4b55b18c16a56566322a', 'request_id': 'req-839e0850-43e3-4ed3-88b9-0ee1879cd7ae'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:22.492094 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-839e0850-43e3-4ed3-88b9-0ee1879cd7ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75b964dfe4ec4b55b18c16a56566322a', 'request_id': 'req-839e0850-43e3-4ed3-88b9-0ee1879cd7ae'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:22.504752 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-839e0850-43e3-4ed3-88b9-0ee1879cd7ae admin admin] The token cache is being invalidate because project 75b964dfe4ec4b55b18c16a56566322a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:36:22.505045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-839e0850-43e3-4ed3-88b9-0ee1879cd7ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:22.505727 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-839e0850-43e3-4ed3-88b9-0ee1879cd7ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:22.507348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1241/2472] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:36:22 2022] DELETE /identity/v3/projects/75b964dfe4ec4b55b18c16a56566322a => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:22.514135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3700b-e85c-47b6-90b5-299bc9381dc3 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:22.514135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3700b-e85c-47b6-90b5-299bc9381dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:22.514135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3700b-e85c-47b6-90b5-299bc9381dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:22.534092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-92e3700b-e85c-47b6-90b5-299bc9381dc3 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:22.535348 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-92e3700b-e85c-47b6-90b5-299bc9381dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:22.570167 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:22.596343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1232/2473] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:22.606117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54302dc-3aa6-4999-a67d-405f10132516 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:22.611526 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54302dc-3aa6-4999-a67d-405f10132516 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:22.611526 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54302dc-3aa6-4999-a67d-405f10132516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:22.627593 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d54302dc-3aa6-4999-a67d-405f10132516 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:22.628835 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d54302dc-3aa6-4999-a67d-405f10132516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:22.637826 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1242/2474] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:22.642038 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3700b-e85c-47b6-90b5-299bc9381dc3 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:22.652131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3700b-e85c-47b6-90b5-299bc9381dc3 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:22.657026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c3810-67d2-4463-a4e6-f179181c3e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:22.657832 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c3810-67d2-4463-a4e6-f179181c3e78 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:22.658403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c3810-67d2-4463-a4e6-f179181c3e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:22.658688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c3810-67d2-4463-a4e6-f179181c3e78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:22.659414 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c3810-67d2-4463-a4e6-f179181c3e78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:36:22.659727 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c3810-67d2-4463-a4e6-f179181c3e78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:22.661367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c3810-67d2-4463-a4e6-f179181c3e78 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:22.673759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1233/2475] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:36:22.678024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54302dc-3aa6-4999-a67d-405f10132516 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:22.688103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54302dc-3aa6-4999-a67d-405f10132516 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:22.693111 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f8f4e-7a03-4c8c-b027-0e2944bc6017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:22.693714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f8f4e-7a03-4c8c-b027-0e2944bc6017 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:22.693996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f8f4e-7a03-4c8c-b027-0e2944bc6017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:22.694272 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f8f4e-7a03-4c8c-b027-0e2944bc6017 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:22.694949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f8f4e-7a03-4c8c-b027-0e2944bc6017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:22.696897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f8f4e-7a03-4c8c-b027-0e2944bc6017 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:22.790452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1243/2476] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:36:22 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:22.794857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c3810-67d2-4463-a4e6-f179181c3e78 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:22.805095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c3810-67d2-4463-a4e6-f179181c3e78 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:22.863789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a3922-dae6-43d6-be98-3d9b37d1e53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:22.865422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a3922-dae6-43d6-be98-3d9b37d1e53d admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:22.865705 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a3922-dae6-43d6-be98-3d9b37d1e53d admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:22.865983 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a3922-dae6-43d6-be98-3d9b37d1e53d admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:22.866636 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a3922-dae6-43d6-be98-3d9b37d1e53d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:22.868618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a3922-dae6-43d6-be98-3d9b37d1e53d admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:22.972976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1234/2477] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:36:22 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:22.977208 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f8f4e-7a03-4c8c-b027-0e2944bc6017 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:22.988801 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f8f4e-7a03-4c8c-b027-0e2944bc6017 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:22.993382 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e97b4-6836-4ae6-ac68-d0a0177aecf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:22.993976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e97b4-6836-4ae6-ac68-d0a0177aecf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:22.994328 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e97b4-6836-4ae6-ac68-d0a0177aecf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:22.995298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e97b4-6836-4ae6-ac68-d0a0177aecf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:22.995976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e97b4-6836-4ae6-ac68-d0a0177aecf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:22.997712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e97b4-6836-4ae6-ac68-d0a0177aecf5 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:23.008196 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1244/2478] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:23.012472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a3922-dae6-43d6-be98-3d9b37d1e53d admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:23.022773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a3922-dae6-43d6-be98-3d9b37d1e53d admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:23.028141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91934ff5-9742-4916-9342-50ab1223ea96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:23.028744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91934ff5-9742-4916-9342-50ab1223ea96 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:23.029030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91934ff5-9742-4916-9342-50ab1223ea96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:23.029306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91934ff5-9742-4916-9342-50ab1223ea96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:23.029969 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91934ff5-9742-4916-9342-50ab1223ea96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:23.031674 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91934ff5-9742-4916-9342-50ab1223ea96 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:23.044793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1235/2479] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:23.047825 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e97b4-6836-4ae6-ac68-d0a0177aecf5 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:23.058803 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e97b4-6836-4ae6-ac68-d0a0177aecf5 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:23.069857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913de70-a205-4b41-b190-467cac810a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:23.069857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913de70-a205-4b41-b190-467cac810a26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:23.069857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913de70-a205-4b41-b190-467cac810a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:23.069857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913de70-a205-4b41-b190-467cac810a26 admin admin] PATH_INFO: `/v3/projects/289075d635074deda951a6bdacbfd27c/users/cde16f0f62ee478ba97945febdb1d7c4/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:23.080869 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913de70-a205-4b41-b190-467cac810a26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=289075d635074deda951a6bdacbfd27c, user_id=cde16f0f62ee478ba97945febdb1d7c4, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:23.083848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913de70-a205-4b41-b190-467cac810a26 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:23.178318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1245/2480] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:36:23 2022] PUT /identity/v3/projects/289075d635074deda951a6bdacbfd27c/users/cde16f0f62ee478ba97945febdb1d7c4/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:23.189476 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062011e-5ef6-4b44-9c4d-69176c3c1b94 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:23.189762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062011e-5ef6-4b44-9c4d-69176c3c1b94 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:23.190043 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062011e-5ef6-4b44-9c4d-69176c3c1b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:23.217992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-0062011e-5ef6-4b44-9c4d-69176c3c1b94 None None] MFA Rules not processed for user `cde16f0f62ee478ba97945febdb1d7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:23.219279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-0062011e-5ef6-4b44-9c4d-69176c3c1b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:23.285941 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1236/2481] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:23.290525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913de70-a205-4b41-b190-467cac810a26 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:23.321992 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:23.357915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913de70-a205-4b41-b190-467cac810a26 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:23.363052 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3933027-eaf1-47fc-a83b-d01fc4374ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:23.365105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3933027-eaf1-47fc-a83b-d01fc4374ade admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:23.365105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3933027-eaf1-47fc-a83b-d01fc4374ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:23.365105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3933027-eaf1-47fc-a83b-d01fc4374ade admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:23.365105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3933027-eaf1-47fc-a83b-d01fc4374ade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:23.366812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3933027-eaf1-47fc-a83b-d01fc4374ade admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:23.500858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1246/2482] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:36:23 2022] POST /identity/v3/projects => generated 387 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:23.505028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062011e-5ef6-4b44-9c4d-69176c3c1b94 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:23.516807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062011e-5ef6-4b44-9c4d-69176c3c1b94 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:23.551867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:23.577088 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea876a-1a25-48a8-99e6-91ce1f18765b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:23.577702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea876a-1a25-48a8-99e6-91ce1f18765b admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:23.577986 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea876a-1a25-48a8-99e6-91ce1f18765b admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:23.578264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea876a-1a25-48a8-99e6-91ce1f18765b admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:23.578949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea876a-1a25-48a8-99e6-91ce1f18765b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:23.580943 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea876a-1a25-48a8-99e6-91ce1f18765b admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:23.691301 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1237/2483] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:36:23 2022] POST /identity/v3/users => generated 300 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:23.695891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3933027-eaf1-47fc-a83b-d01fc4374ade admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:23.706930 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3933027-eaf1-47fc-a83b-d01fc4374ade admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:23.711888 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94284a-deb6-4443-b3fe-54b28ff12649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:23.712489 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94284a-deb6-4443-b3fe-54b28ff12649 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:23.712768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94284a-deb6-4443-b3fe-54b28ff12649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:23.713039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94284a-deb6-4443-b3fe-54b28ff12649 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:23.714242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94284a-deb6-4443-b3fe-54b28ff12649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:23.715879 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94284a-deb6-4443-b3fe-54b28ff12649 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:23.725949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1247/2484] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:36:23.730286 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea876a-1a25-48a8-99e6-91ce1f18765b admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:23.740747 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea876a-1a25-48a8-99e6-91ce1f18765b admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:23.745309 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c631d7-ae6a-437e-97e3-057067df01a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:23.745966 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c631d7-ae6a-437e-97e3-057067df01a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:23.746244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c631d7-ae6a-437e-97e3-057067df01a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:23.746517 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c631d7-ae6a-437e-97e3-057067df01a6 admin admin] PATH_INFO: `/v3/projects/bedc8df3295444c68634c5538d76eb49/users/f6c678e7d3894600b6cbe68a90adc96b/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:23.762071 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c631d7-ae6a-437e-97e3-057067df01a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bedc8df3295444c68634c5538d76eb49, user_id=f6c678e7d3894600b6cbe68a90adc96b, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:23.764784 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c631d7-ae6a-437e-97e3-057067df01a6 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:23.852067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1238/2485] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:36:23 2022] PUT /identity/v3/projects/bedc8df3295444c68634c5538d76eb49/users/f6c678e7d3894600b6cbe68a90adc96b/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:23.856683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94284a-deb6-4443-b3fe-54b28ff12649 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:23.866351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94284a-deb6-4443-b3fe-54b28ff12649 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:23.923490 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb75c91-22ab-456d-ab3e-45cd3e5348d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:23.924095 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb75c91-22ab-456d-ab3e-45cd3e5348d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:23.924624 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb75c91-22ab-456d-ab3e-45cd3e5348d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:23.924818 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb75c91-22ab-456d-ab3e-45cd3e5348d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:23.925460 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb75c91-22ab-456d-ab3e-45cd3e5348d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:23.927094 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb75c91-22ab-456d-ab3e-45cd3e5348d1 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:23.936997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1248/2486] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:23.941432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c631d7-ae6a-437e-97e3-057067df01a6 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:23.952545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c631d7-ae6a-437e-97e3-057067df01a6 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:23.957161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57faed42-777e-4bb5-9bf4-c3c204a1ec70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:23.957805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faed42-777e-4bb5-9bf4-c3c204a1ec70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:23.958354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faed42-777e-4bb5-9bf4-c3c204a1ec70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:23.958608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faed42-777e-4bb5-9bf4-c3c204a1ec70 admin admin] PATH_INFO: `/v3/projects/bedc8df3295444c68634c5538d76eb49/users/f6c678e7d3894600b6cbe68a90adc96b/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:23.960516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57faed42-777e-4bb5-9bf4-c3c204a1ec70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bedc8df3295444c68634c5538d76eb49, user_id=f6c678e7d3894600b6cbe68a90adc96b, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:23.963330 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57faed42-777e-4bb5-9bf4-c3c204a1ec70 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:23.975677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1239/2487] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:36:23 2022] PUT /identity/v3/projects/bedc8df3295444c68634c5538d76eb49/users/f6c678e7d3894600b6cbe68a90adc96b/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:23.985861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2f85d-47ac-4df4-b680-669e22ce3689 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:23.986149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2f85d-47ac-4df4-b680-669e22ce3689 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:23.986423 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2f85d-47ac-4df4-b680-669e22ce3689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:24.015259 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-8cb2f85d-47ac-4df4-b680-669e22ce3689 None None] MFA Rules not processed for user `f6c678e7d3894600b6cbe68a90adc96b`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:24.016463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8cb2f85d-47ac-4df4-b680-669e22ce3689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:24.084181 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1249/2488] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:24.095501 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57faed42-777e-4bb5-9bf4-c3c204a1ec70 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:24.097338 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-57faed42-777e-4bb5-9bf4-c3c204a1ec70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:24.159363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57faed42-777e-4bb5-9bf4-c3c204a1ec70 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:24.162977 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200fefd-2282-4c48-8705-785fb469523a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:24.163724 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200fefd-2282-4c48-8705-785fb469523a service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:24.164073 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200fefd-2282-4c48-8705-785fb469523a service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:24.164332 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200fefd-2282-4c48-8705-785fb469523a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:24.173974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d200fefd-2282-4c48-8705-785fb469523a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:24.175440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d200fefd-2282-4c48-8705-785fb469523a service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:24.185516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1240/2489] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:24.214829 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2f85d-47ac-4df4-b680-669e22ce3689 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:24.239844 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:24.277997 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2f85d-47ac-4df4-b680-669e22ce3689 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:24.283310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b461469-7944-4d2b-872e-c49006ffd31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:24.284036 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b461469-7944-4d2b-872e-c49006ffd31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:24.284325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b461469-7944-4d2b-872e-c49006ffd31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:24.284603 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b461469-7944-4d2b-872e-c49006ffd31f admin admin] PATH_INFO: `/v3/users/cde16f0f62ee478ba97945febdb1d7c4` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:24.285942 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b461469-7944-4d2b-872e-c49006ffd31f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cde16f0f62ee478ba97945febdb1d7c4)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:24.287993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b461469-7944-4d2b-872e-c49006ffd31f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:24.528354 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-8b461469-7944-4d2b-872e-c49006ffd31f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cde16f0f62ee478ba97945febdb1d7c4'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:24.540283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-8b461469-7944-4d2b-872e-c49006ffd31f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cde16f0f62ee478ba97945febdb1d7c4'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:24.611430 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-8b461469-7944-4d2b-872e-c49006ffd31f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cde16f0f62ee478ba97945febdb1d7c4'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:24.618935 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-8b461469-7944-4d2b-872e-c49006ffd31f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cde16f0f62ee478ba97945febdb1d7c4'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:24.621729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-8b461469-7944-4d2b-872e-c49006ffd31f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cde16f0f62ee478ba97945febdb1d7c4'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:24.626269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1250/2490] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:36:24 2022] DELETE /identity/v3/users/cde16f0f62ee478ba97945febdb1d7c4 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:24.630802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae777f-4cb4-414f-bdc1-12a38ce8bd88 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:24.631333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae777f-4cb4-414f-bdc1-12a38ce8bd88 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:24.631445 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae777f-4cb4-414f-bdc1-12a38ce8bd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:24.651365 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-78ae777f-4cb4-414f-bdc1-12a38ce8bd88 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:24.652217 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-78ae777f-4cb4-414f-bdc1-12a38ce8bd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:24.712852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1241/2491] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:24.722902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b461469-7944-4d2b-872e-c49006ffd31f admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:24.724357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8b461469-7944-4d2b-872e-c49006ffd31f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:24.784330 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b461469-7944-4d2b-872e-c49006ffd31f admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:24.787808 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6efffc-874a-4279-98f8-589fd518e07a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:24.788413 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6efffc-874a-4279-98f8-589fd518e07a service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:24.788687 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6efffc-874a-4279-98f8-589fd518e07a service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:24.788955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6efffc-874a-4279-98f8-589fd518e07a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:24.798010 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6efffc-874a-4279-98f8-589fd518e07a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:24.799525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6efffc-874a-4279-98f8-589fd518e07a service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:24.809306 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1251/2492] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:25.253812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae777f-4cb4-414f-bdc1-12a38ce8bd88 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:25.255682 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-78ae777f-4cb4-414f-bdc1-12a38ce8bd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:25.268592 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae777f-4cb4-414f-bdc1-12a38ce8bd88 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:25.276914 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:25.278270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:25.278775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:25.279139 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a admin admin] PATH_INFO: `/v3/projects/289075d635074deda951a6bdacbfd27c` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:25.280527 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=289075d635074deda951a6bdacbfd27c)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:25.282659 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:25.373495 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:25.390156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '289075d635074deda951a6bdacbfd27c', 'request_id': 'req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:25.395883 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '289075d635074deda951a6bdacbfd27c', 'request_id': 'req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:25.402230 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '289075d635074deda951a6bdacbfd27c', 'request_id': 'req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:25.486522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a admin admin] The token cache is being invalidate because project 289075d635074deda951a6bdacbfd27c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:36:25.486815 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:25.487482 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:25.488980 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1242/2493] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:36:25 2022] DELETE /identity/v3/projects/289075d635074deda951a6bdacbfd27c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:25.492995 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6efffc-874a-4279-98f8-589fd518e07a service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:25.503635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6efffc-874a-4279-98f8-589fd518e07a service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:25.553785 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:25.568443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4666b83-4cf5-46ab-8f1a-db6d825c2e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:25.569075 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4666b83-4cf5-46ab-8f1a-db6d825c2e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:25.569356 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4666b83-4cf5-46ab-8f1a-db6d825c2e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:25.569633 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4666b83-4cf5-46ab-8f1a-db6d825c2e15 admin admin] PATH_INFO: `/v3/users/f6c678e7d3894600b6cbe68a90adc96b` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:25.570758 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4666b83-4cf5-46ab-8f1a-db6d825c2e15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6c678e7d3894600b6cbe68a90adc96b)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:25.572737 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4666b83-4cf5-46ab-8f1a-db6d825c2e15 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:25.849279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a4666b83-4cf5-46ab-8f1a-db6d825c2e15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6c678e7d3894600b6cbe68a90adc96b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:25.863641 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a4666b83-4cf5-46ab-8f1a-db6d825c2e15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6c678e7d3894600b6cbe68a90adc96b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:25.942393 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a4666b83-4cf5-46ab-8f1a-db6d825c2e15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6c678e7d3894600b6cbe68a90adc96b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:25.950497 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a4666b83-4cf5-46ab-8f1a-db6d825c2e15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6c678e7d3894600b6cbe68a90adc96b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:25.951161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a4666b83-4cf5-46ab-8f1a-db6d825c2e15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6c678e7d3894600b6cbe68a90adc96b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:25.957231 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1252/2494] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:36:25 2022] DELETE /identity/v3/users/f6c678e7d3894600b6cbe68a90adc96b => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:26.419150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:26.420703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:26.435025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a89f1b-4ff6-4a4e-a399-51a3f560db8a admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:26.506468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e69ad4-990b-4903-b4e7-7311580ad9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:26.507134 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e69ad4-990b-4903-b4e7-7311580ad9c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:26.507418 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e69ad4-990b-4903-b4e7-7311580ad9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:26.507695 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e69ad4-990b-4903-b4e7-7311580ad9c9 admin admin] PATH_INFO: `/v3/projects/bedc8df3295444c68634c5538d76eb49` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:26.508822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e69ad4-990b-4903-b4e7-7311580ad9c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bedc8df3295444c68634c5538d76eb49)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:26.511474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e69ad4-990b-4903-b4e7-7311580ad9c9 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:26.658600 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a9e69ad4-990b-4903-b4e7-7311580ad9c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bedc8df3295444c68634c5538d76eb49', 'request_id': 'req-a9e69ad4-990b-4903-b4e7-7311580ad9c9'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:26.664177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a9e69ad4-990b-4903-b4e7-7311580ad9c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bedc8df3295444c68634c5538d76eb49', 'request_id': 'req-a9e69ad4-990b-4903-b4e7-7311580ad9c9'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:26.670930 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a9e69ad4-990b-4903-b4e7-7311580ad9c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bedc8df3295444c68634c5538d76eb49', 'request_id': 'req-a9e69ad4-990b-4903-b4e7-7311580ad9c9'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:26.749444 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a9e69ad4-990b-4903-b4e7-7311580ad9c9 admin admin] The token cache is being invalidate because project bedc8df3295444c68634c5538d76eb49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:36:26.749742 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a9e69ad4-990b-4903-b4e7-7311580ad9c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:26.750378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a9e69ad4-990b-4903-b4e7-7311580ad9c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:26.751902 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1243/2495] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:36:26 2022] DELETE /identity/v3/projects/bedc8df3295444c68634c5538d76eb49 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:26.756870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3192d56-d467-4b9e-b6f8-4c464a1824b5 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:26.757420 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3192d56-d467-4b9e-b6f8-4c464a1824b5 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:26.757529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3192d56-d467-4b9e-b6f8-4c464a1824b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:26.777404 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a3192d56-d467-4b9e-b6f8-4c464a1824b5 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:26.778632 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a3192d56-d467-4b9e-b6f8-4c464a1824b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:26.838626 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1253/2496] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:26.847536 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c620fd-8ab7-42af-9e30-238ad22e4709 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:26.847824 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c620fd-8ab7-42af-9e30-238ad22e4709 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:26.848095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c620fd-8ab7-42af-9e30-238ad22e4709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:26.870101 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-55c620fd-8ab7-42af-9e30-238ad22e4709 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:26.871400 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-55c620fd-8ab7-42af-9e30-238ad22e4709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:26.879570 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1244/2497] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:26.883949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3192d56-d467-4b9e-b6f8-4c464a1824b5 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:26.893803 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3192d56-d467-4b9e-b6f8-4c464a1824b5 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:26.899391 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d931fe-2c28-4b63-9828-3b6f864f8527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:26.899990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d931fe-2c28-4b63-9828-3b6f864f8527 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:26.900272 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d931fe-2c28-4b63-9828-3b6f864f8527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:26.900575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d931fe-2c28-4b63-9828-3b6f864f8527 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:26.901229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d931fe-2c28-4b63-9828-3b6f864f8527 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:36:26.901529 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d931fe-2c28-4b63-9828-3b6f864f8527 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:26.903159 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d931fe-2c28-4b63-9828-3b6f864f8527 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:26.914719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1254/2498] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:36:26.919733 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c620fd-8ab7-42af-9e30-238ad22e4709 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:26.928998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c620fd-8ab7-42af-9e30-238ad22e4709 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:26.934342 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dd539-76b2-4409-8a80-4e495e542ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:26.934968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dd539-76b2-4409-8a80-4e495e542ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:26.935250 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dd539-76b2-4409-8a80-4e495e542ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:26.935522 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dd539-76b2-4409-8a80-4e495e542ba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:26.936146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45dd539-76b2-4409-8a80-4e495e542ba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:26.938043 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45dd539-76b2-4409-8a80-4e495e542ba3 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:27.031689 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1245/2499] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:36:26 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:27.035864 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d931fe-2c28-4b63-9828-3b6f864f8527 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:27.046522 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d931fe-2c28-4b63-9828-3b6f864f8527 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:27.094394 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:27.104316 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a2a337-4cb5-4745-b2fc-e0638e28f8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:27.105469 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2a337-4cb5-4745-b2fc-e0638e28f8bf admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:27.105752 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2a337-4cb5-4745-b2fc-e0638e28f8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:27.106031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2a337-4cb5-4745-b2fc-e0638e28f8bf admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:27.106662 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a2a337-4cb5-4745-b2fc-e0638e28f8bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:27.108699 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a2a337-4cb5-4745-b2fc-e0638e28f8bf admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:27.286388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1255/2500] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:36:27 2022] POST /identity/v3/users => generated 294 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:27.290554 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dd539-76b2-4409-8a80-4e495e542ba3 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:27.300968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dd539-76b2-4409-8a80-4e495e542ba3 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:27.305485 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d3a7c-e5f3-4053-9b50-30e3da535a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:27.306079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d3a7c-e5f3-4053-9b50-30e3da535a87 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:27.306359 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d3a7c-e5f3-4053-9b50-30e3da535a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:27.306627 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d3a7c-e5f3-4053-9b50-30e3da535a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:27.308047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d3a7c-e5f3-4053-9b50-30e3da535a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:27.309622 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d3a7c-e5f3-4053-9b50-30e3da535a87 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:27.319780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1246/2501] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:27.324730 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a2a337-4cb5-4745-b2fc-e0638e28f8bf admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:27.335274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a2a337-4cb5-4745-b2fc-e0638e28f8bf admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:27.340166 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3130fe5-c805-44cb-9397-ea8afcb59299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:27.340776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3130fe5-c805-44cb-9397-ea8afcb59299 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:27.341068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3130fe5-c805-44cb-9397-ea8afcb59299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:27.341345 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3130fe5-c805-44cb-9397-ea8afcb59299 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:27.341996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3130fe5-c805-44cb-9397-ea8afcb59299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:27.343636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3130fe5-c805-44cb-9397-ea8afcb59299 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:27.355018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1256/2502] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:27.359596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d3a7c-e5f3-4053-9b50-30e3da535a87 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:27.366164 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:27.370675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d3a7c-e5f3-4053-9b50-30e3da535a87 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:27.375389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b6be9-988a-4c29-af2c-6a9c950dd38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:27.378151 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b6be9-988a-4c29-af2c-6a9c950dd38f admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:27.378151 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b6be9-988a-4c29-af2c-6a9c950dd38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:27.378151 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b6be9-988a-4c29-af2c-6a9c950dd38f admin admin] PATH_INFO: `/v3/projects/ab5814dbfc454373bdda2a16fa7e58f3/users/eff97383418946118a39276a04d161c2/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:27.392422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b6be9-988a-4c29-af2c-6a9c950dd38f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab5814dbfc454373bdda2a16fa7e58f3, user_id=eff97383418946118a39276a04d161c2, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:27.395207 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b6be9-988a-4c29-af2c-6a9c950dd38f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:27.481592 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1247/2503] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:36:27 2022] PUT /identity/v3/projects/ab5814dbfc454373bdda2a16fa7e58f3/users/eff97383418946118a39276a04d161c2/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:27.492543 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562d555-88e3-4a3e-9ca3-a247a2391c55 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:27.492834 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562d555-88e3-4a3e-9ca3-a247a2391c55 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:27.493112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562d555-88e3-4a3e-9ca3-a247a2391c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:27.524617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-1562d555-88e3-4a3e-9ca3-a247a2391c55 None None] MFA Rules not processed for user `eff97383418946118a39276a04d161c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:27.526054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-1562d555-88e3-4a3e-9ca3-a247a2391c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:27.597143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1257/2504] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:36:27.601658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b6be9-988a-4c29-af2c-6a9c950dd38f admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:27.665801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b6be9-988a-4c29-af2c-6a9c950dd38f admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:27.671387 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9fca7-bcc8-4823-83cc-c1a1f8b7a780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:27.672015 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9fca7-bcc8-4823-83cc-c1a1f8b7a780 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:27.672296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9fca7-bcc8-4823-83cc-c1a1f8b7a780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:27.672570 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9fca7-bcc8-4823-83cc-c1a1f8b7a780 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:27.673226 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d9fca7-bcc8-4823-83cc-c1a1f8b7a780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:27.675169 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d9fca7-bcc8-4823-83cc-c1a1f8b7a780 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:27.786270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1248/2505] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:36:27 2022] POST /identity/v3/projects => generated 387 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:27.790792 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562d555-88e3-4a3e-9ca3-a247a2391c55 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:27.803157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562d555-88e3-4a3e-9ca3-a247a2391c55 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:27.861892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f87728-2ca0-48f1-90d2-5b54ecc07fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:27.862511 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f87728-2ca0-48f1-90d2-5b54ecc07fa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:27.862806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f87728-2ca0-48f1-90d2-5b54ecc07fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:27.863126 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f87728-2ca0-48f1-90d2-5b54ecc07fa2 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:27.863739 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f87728-2ca0-48f1-90d2-5b54ecc07fa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:27.865666 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f87728-2ca0-48f1-90d2-5b54ecc07fa2 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:28.012395 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1258/2506] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:36:27 2022] POST /identity/v3/users => generated 300 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:28.018391 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9fca7-bcc8-4823-83cc-c1a1f8b7a780 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:28.031189 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9fca7-bcc8-4823-83cc-c1a1f8b7a780 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:28.035949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05a6ea-5917-4591-8ebd-bf8e87f101ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:28.036658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05a6ea-5917-4591-8ebd-bf8e87f101ac admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:28.036886 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05a6ea-5917-4591-8ebd-bf8e87f101ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:28.037156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05a6ea-5917-4591-8ebd-bf8e87f101ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:28.037796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05a6ea-5917-4591-8ebd-bf8e87f101ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:28.039579 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05a6ea-5917-4591-8ebd-bf8e87f101ac admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:28.052583 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1249/2507] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:28.057253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f87728-2ca0-48f1-90d2-5b54ecc07fa2 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:28.067594 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f87728-2ca0-48f1-90d2-5b54ecc07fa2 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:28.072528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdafd7-217f-4806-a219-b3acd17fbb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:28.073723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdafd7-217f-4806-a219-b3acd17fbb9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:28.073881 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdafd7-217f-4806-a219-b3acd17fbb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:28.074120 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdafd7-217f-4806-a219-b3acd17fbb9a admin admin] PATH_INFO: `/v3/projects/dad1675163b94e1894066b64f27e4db3/users/e21357945ef1490ab9d4c397d19d48ce/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:28.091351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdafd7-217f-4806-a219-b3acd17fbb9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad1675163b94e1894066b64f27e4db3, user_id=e21357945ef1490ab9d4c397d19d48ce, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:28.094045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdafd7-217f-4806-a219-b3acd17fbb9a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:28.167325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1259/2508] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:36:28 2022] PUT /identity/v3/projects/dad1675163b94e1894066b64f27e4db3/users/e21357945ef1490ab9d4c397d19d48ce/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:28.171335 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05a6ea-5917-4591-8ebd-bf8e87f101ac admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:28.182068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05a6ea-5917-4591-8ebd-bf8e87f101ac admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:28.241018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bfe9b-ee13-4734-a37a-1471c7f18642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:28.241673 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bfe9b-ee13-4734-a37a-1471c7f18642 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:28.241956 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bfe9b-ee13-4734-a37a-1471c7f18642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:28.242233 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bfe9b-ee13-4734-a37a-1471c7f18642 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:28.242893 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bfe9b-ee13-4734-a37a-1471c7f18642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:28.244968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bfe9b-ee13-4734-a37a-1471c7f18642 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:28.255289 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1250/2509] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:28.259654 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdafd7-217f-4806-a219-b3acd17fbb9a admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:28.271034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdafd7-217f-4806-a219-b3acd17fbb9a admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:28.276790 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2eade-e20a-41a0-a754-990585497007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:28.277378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2eade-e20a-41a0-a754-990585497007 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:28.277665 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2eade-e20a-41a0-a754-990585497007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:28.277941 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2eade-e20a-41a0-a754-990585497007 admin admin] PATH_INFO: `/v3/projects/dad1675163b94e1894066b64f27e4db3/users/e21357945ef1490ab9d4c397d19d48ce/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:28.279965 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2eade-e20a-41a0-a754-990585497007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad1675163b94e1894066b64f27e4db3, user_id=e21357945ef1490ab9d4c397d19d48ce, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:28.282679 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2eade-e20a-41a0-a754-990585497007 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:28.295424 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1260/2510] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:36:28 2022] PUT /identity/v3/projects/dad1675163b94e1894066b64f27e4db3/users/e21357945ef1490ab9d4c397d19d48ce/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:28.306754 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74787039-b096-4d78-8b30-22dae3599a74 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:28.311820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74787039-b096-4d78-8b30-22dae3599a74 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:28.311820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74787039-b096-4d78-8b30-22dae3599a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:28.336896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-74787039-b096-4d78-8b30-22dae3599a74 None None] MFA Rules not processed for user `e21357945ef1490ab9d4c397d19d48ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:28.338848 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-74787039-b096-4d78-8b30-22dae3599a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:28.379391 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:28.408468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1251/2511] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:28 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:28.419598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2eade-e20a-41a0-a754-990585497007 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:28.421373 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-2cc2eade-e20a-41a0-a754-990585497007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:28.436623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:28.481653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2eade-e20a-41a0-a754-990585497007 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:28.485352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291ba90-d35f-4a3a-abcf-64e9d7a2a18d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:28.485976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291ba90-d35f-4a3a-abcf-64e9d7a2a18d service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:28.486254 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291ba90-d35f-4a3a-abcf-64e9d7a2a18d service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:28.486528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291ba90-d35f-4a3a-abcf-64e9d7a2a18d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:28.496091 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0291ba90-d35f-4a3a-abcf-64e9d7a2a18d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:28.497513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0291ba90-d35f-4a3a-abcf-64e9d7a2a18d service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:28.507333 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1261/2512] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:28 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:28.651913 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74787039-b096-4d78-8b30-22dae3599a74 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:28.718635 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74787039-b096-4d78-8b30-22dae3599a74 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:28.723296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bfd51-2b62-4aac-ae4a-a8db49d98d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:28.723939 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bfd51-2b62-4aac-ae4a-a8db49d98d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:28.724224 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bfd51-2b62-4aac-ae4a-a8db49d98d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:28.724501 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bfd51-2b62-4aac-ae4a-a8db49d98d7e admin admin] PATH_INFO: `/v3/users/eff97383418946118a39276a04d161c2` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:28.725799 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bfd51-2b62-4aac-ae4a-a8db49d98d7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eff97383418946118a39276a04d161c2)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:28.727787 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bfd51-2b62-4aac-ae4a-a8db49d98d7e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:28.917911 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-cb4bfd51-2b62-4aac-ae4a-a8db49d98d7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eff97383418946118a39276a04d161c2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:29.005832 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-cb4bfd51-2b62-4aac-ae4a-a8db49d98d7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eff97383418946118a39276a04d161c2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:29.006490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-cb4bfd51-2b62-4aac-ae4a-a8db49d98d7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eff97383418946118a39276a04d161c2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:29.019965 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-cb4bfd51-2b62-4aac-ae4a-a8db49d98d7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eff97383418946118a39276a04d161c2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:29.020908 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-cb4bfd51-2b62-4aac-ae4a-a8db49d98d7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eff97383418946118a39276a04d161c2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:29.034181 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1252/2513] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:36:28 2022] DELETE /identity/v3/users/eff97383418946118a39276a04d161c2 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:36:29.038722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4be454-9354-4f63-9f9f-5ca8e3f3ecae None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:29.039235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4be454-9354-4f63-9f9f-5ca8e3f3ecae None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:29.039380 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4be454-9354-4f63-9f9f-5ca8e3f3ecae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:29.059644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-6c4be454-9354-4f63-9f9f-5ca8e3f3ecae None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:29.060886 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6c4be454-9354-4f63-9f9f-5ca8e3f3ecae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:29.124823 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1262/2514] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:29.135076 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bfd51-2b62-4aac-ae4a-a8db49d98d7e admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:29.136573 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-cb4bfd51-2b62-4aac-ae4a-a8db49d98d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:29.197791 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bfd51-2b62-4aac-ae4a-a8db49d98d7e admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:29.201645 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5255848-92a3-41b5-8562-f37f8a2e0180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:29.202269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5255848-92a3-41b5-8562-f37f8a2e0180 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:29.202552 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5255848-92a3-41b5-8562-f37f8a2e0180 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:29.202831 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5255848-92a3-41b5-8562-f37f8a2e0180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:29.213052 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5255848-92a3-41b5-8562-f37f8a2e0180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:29.214516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5255848-92a3-41b5-8562-f37f8a2e0180 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:29.218620 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:29.224715 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1253/2515] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:29.670084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4be454-9354-4f63-9f9f-5ca8e3f3ecae None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:29.671760 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6c4be454-9354-4f63-9f9f-5ca8e3f3ecae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:29.684416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4be454-9354-4f63-9f9f-5ca8e3f3ecae None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:29.689094 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ddca3-8258-4470-811d-4aec03f23e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:29.689744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ddca3-8258-4470-811d-4aec03f23e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:29.690024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ddca3-8258-4470-811d-4aec03f23e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:29.690367 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ddca3-8258-4470-811d-4aec03f23e73 admin admin] PATH_INFO: `/v3/projects/ab5814dbfc454373bdda2a16fa7e58f3` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:29.691507 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ddca3-8258-4470-811d-4aec03f23e73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab5814dbfc454373bdda2a16fa7e58f3)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:29.693564 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ddca3-8258-4470-811d-4aec03f23e73 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:29.799775 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a66ddca3-8258-4470-811d-4aec03f23e73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab5814dbfc454373bdda2a16fa7e58f3', 'request_id': 'req-a66ddca3-8258-4470-811d-4aec03f23e73'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:29.805231 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a66ddca3-8258-4470-811d-4aec03f23e73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab5814dbfc454373bdda2a16fa7e58f3', 'request_id': 'req-a66ddca3-8258-4470-811d-4aec03f23e73'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:29.868955 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a66ddca3-8258-4470-811d-4aec03f23e73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab5814dbfc454373bdda2a16fa7e58f3', 'request_id': 'req-a66ddca3-8258-4470-811d-4aec03f23e73'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:29.881503 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a66ddca3-8258-4470-811d-4aec03f23e73 admin admin] The token cache is being invalidate because project ab5814dbfc454373bdda2a16fa7e58f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:36:29.881794 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a66ddca3-8258-4470-811d-4aec03f23e73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:29.882446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a66ddca3-8258-4470-811d-4aec03f23e73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:29.883972 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1263/2516] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:36:29 2022] DELETE /identity/v3/projects/ab5814dbfc454373bdda2a16fa7e58f3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:29.888716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5255848-92a3-41b5-8562-f37f8a2e0180 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:29.898738 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5255848-92a3-41b5-8562-f37f8a2e0180 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:29.958221 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6f514-ba8a-4197-9b6e-c01d8a62658e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:29.958858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6f514-ba8a-4197-9b6e-c01d8a62658e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:29.959161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6f514-ba8a-4197-9b6e-c01d8a62658e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:29.959434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6f514-ba8a-4197-9b6e-c01d8a62658e admin admin] PATH_INFO: `/v3/users/e21357945ef1490ab9d4c397d19d48ce` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:29.960568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6f514-ba8a-4197-9b6e-c01d8a62658e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e21357945ef1490ab9d4c397d19d48ce)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:29.962473 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6f514-ba8a-4197-9b6e-c01d8a62658e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:30.161279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-bcc6f514-ba8a-4197-9b6e-c01d8a62658e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e21357945ef1490ab9d4c397d19d48ce'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:30.234693 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-bcc6f514-ba8a-4197-9b6e-c01d8a62658e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e21357945ef1490ab9d4c397d19d48ce'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:30.235399 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-bcc6f514-ba8a-4197-9b6e-c01d8a62658e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e21357945ef1490ab9d4c397d19d48ce'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:30.252985 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-bcc6f514-ba8a-4197-9b6e-c01d8a62658e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e21357945ef1490ab9d4c397d19d48ce'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:30.253719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-bcc6f514-ba8a-4197-9b6e-c01d8a62658e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e21357945ef1490ab9d4c397d19d48ce'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:30.269338 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1254/2517] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:36:29 2022] DELETE /identity/v3/users/e21357945ef1490ab9d4c397d19d48ce => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:36:30.808670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ddca3-8258-4470-811d-4aec03f23e73 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:30.810605 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a66ddca3-8258-4470-811d-4aec03f23e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:30.825958 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ddca3-8258-4470-811d-4aec03f23e73 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:30.867865 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:30.892742 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f285e-86bb-4d07-ae53-313ba946e8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:30.893390 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f285e-86bb-4d07-ae53-313ba946e8ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:30.893678 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f285e-86bb-4d07-ae53-313ba946e8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:30.894033 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f285e-86bb-4d07-ae53-313ba946e8ab admin admin] PATH_INFO: `/v3/projects/dad1675163b94e1894066b64f27e4db3` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:30.895151 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f285e-86bb-4d07-ae53-313ba946e8ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=dad1675163b94e1894066b64f27e4db3)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:30.897196 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f285e-86bb-4d07-ae53-313ba946e8ab admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:31.001947 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-9d8f285e-86bb-4d07-ae53-313ba946e8ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dad1675163b94e1894066b64f27e4db3', 'request_id': 'req-9d8f285e-86bb-4d07-ae53-313ba946e8ab'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:31.008718 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-9d8f285e-86bb-4d07-ae53-313ba946e8ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dad1675163b94e1894066b64f27e4db3', 'request_id': 'req-9d8f285e-86bb-4d07-ae53-313ba946e8ab'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:31.103104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-9d8f285e-86bb-4d07-ae53-313ba946e8ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dad1675163b94e1894066b64f27e4db3', 'request_id': 'req-9d8f285e-86bb-4d07-ae53-313ba946e8ab'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:31.116098 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-9d8f285e-86bb-4d07-ae53-313ba946e8ab admin admin] The token cache is being invalidate because project dad1675163b94e1894066b64f27e4db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:36:31.116421 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-9d8f285e-86bb-4d07-ae53-313ba946e8ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:31.117163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-9d8f285e-86bb-4d07-ae53-313ba946e8ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:31.119391 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1264/2518] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:36:30 2022] DELETE /identity/v3/projects/dad1675163b94e1894066b64f27e4db3 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:31.124424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8494db2-bfd4-4dae-8c76-cab61d2542ac None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:31.124942 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8494db2-bfd4-4dae-8c76-cab61d2542ac None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:31.125041 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8494db2-bfd4-4dae-8c76-cab61d2542ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:31.146431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-c8494db2-bfd4-4dae-8c76-cab61d2542ac None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:31.148590 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c8494db2-bfd4-4dae-8c76-cab61d2542ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:31.211844 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1255/2519] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:31.220974 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7b52b-0420-4bfd-9d65-a663825e6f47 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:31.221264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7b52b-0420-4bfd-9d65-a663825e6f47 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:31.221542 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7b52b-0420-4bfd-9d65-a663825e6f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:31.241447 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-55e7b52b-0420-4bfd-9d65-a663825e6f47 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:31.242668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-55e7b52b-0420-4bfd-9d65-a663825e6f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:31.250710 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1265/2520] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:31.255385 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8494db2-bfd4-4dae-8c76-cab61d2542ac None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:31.266084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8494db2-bfd4-4dae-8c76-cab61d2542ac None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:31.272819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf45261-8898-46a7-9b22-40f1b77d2e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:31.273627 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf45261-8898-46a7-9b22-40f1b77d2e32 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:31.273908 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf45261-8898-46a7-9b22-40f1b77d2e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:31.274229 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf45261-8898-46a7-9b22-40f1b77d2e32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:31.275012 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf45261-8898-46a7-9b22-40f1b77d2e32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:36:31.275494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf45261-8898-46a7-9b22-40f1b77d2e32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:31.277800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf45261-8898-46a7-9b22-40f1b77d2e32 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:31.290716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1256/2521] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:36:31.295231 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7b52b-0420-4bfd-9d65-a663825e6f47 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:31.304578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7b52b-0420-4bfd-9d65-a663825e6f47 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:31.309274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f5d3d-8cc4-4ec0-8e2d-16a06ff43f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:31.309874 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f5d3d-8cc4-4ec0-8e2d-16a06ff43f61 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:31.310154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f5d3d-8cc4-4ec0-8e2d-16a06ff43f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:31.310426 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f5d3d-8cc4-4ec0-8e2d-16a06ff43f61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:31.311084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f5d3d-8cc4-4ec0-8e2d-16a06ff43f61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:31.312972 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f5d3d-8cc4-4ec0-8e2d-16a06ff43f61 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:31.398606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1266/2522] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:36:31 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:31.403798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf45261-8898-46a7-9b22-40f1b77d2e32 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:31.410457 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:31.414286 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf45261-8898-46a7-9b22-40f1b77d2e32 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:31.476114 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d230c-0f23-47bb-b0a5-1b4a468413e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:31.476838 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d230c-0f23-47bb-b0a5-1b4a468413e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:31.477117 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d230c-0f23-47bb-b0a5-1b4a468413e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:31.477394 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d230c-0f23-47bb-b0a5-1b4a468413e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:31.478031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d230c-0f23-47bb-b0a5-1b4a468413e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:31.480605 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d230c-0f23-47bb-b0a5-1b4a468413e3 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:31.567211 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1257/2523] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:36:31 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:31.572976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f5d3d-8cc4-4ec0-8e2d-16a06ff43f61 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:31.590964 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f5d3d-8cc4-4ec0-8e2d-16a06ff43f61 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:31.595768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c7970-16ae-461e-8e33-4861378fe9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:31.596491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c7970-16ae-461e-8e33-4861378fe9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:31.596659 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c7970-16ae-461e-8e33-4861378fe9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:31.596935 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c7970-16ae-461e-8e33-4861378fe9b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:31.597581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c7970-16ae-461e-8e33-4861378fe9b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:31.599305 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c7970-16ae-461e-8e33-4861378fe9b1 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:31.610717 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1267/2524] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:31.615051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d230c-0f23-47bb-b0a5-1b4a468413e3 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:31.626236 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d230c-0f23-47bb-b0a5-1b4a468413e3 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:31.631439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e4f03-b8d1-4a68-83fc-020786e17231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:31.632051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e4f03-b8d1-4a68-83fc-020786e17231 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:31.632338 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e4f03-b8d1-4a68-83fc-020786e17231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:31.632640 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e4f03-b8d1-4a68-83fc-020786e17231 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:31.633290 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e4f03-b8d1-4a68-83fc-020786e17231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:31.635591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e4f03-b8d1-4a68-83fc-020786e17231 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:31.646724 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1258/2525] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:31.651540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c7970-16ae-461e-8e33-4861378fe9b1 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:31.662077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c7970-16ae-461e-8e33-4861378fe9b1 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:31.666976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0046812-8ca9-4cb9-be76-5092e0c553e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:31.667635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0046812-8ca9-4cb9-be76-5092e0c553e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:31.667917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0046812-8ca9-4cb9-be76-5092e0c553e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:31.668194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0046812-8ca9-4cb9-be76-5092e0c553e1 admin admin] PATH_INFO: `/v3/projects/fbccb26141674211a27732b014667ed7/users/e8fc5c19560f4f5ea3cd1390e2c9bff5/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:31.684042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0046812-8ca9-4cb9-be76-5092e0c553e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbccb26141674211a27732b014667ed7, user_id=e8fc5c19560f4f5ea3cd1390e2c9bff5, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:31.686811 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0046812-8ca9-4cb9-be76-5092e0c553e1 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:31.783318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1268/2526] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:36:31 2022] PUT /identity/v3/projects/fbccb26141674211a27732b014667ed7/users/e8fc5c19560f4f5ea3cd1390e2c9bff5/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:31.794081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9780dd-41d6-474d-b1f1-aed34bd55802 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:31.794398 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9780dd-41d6-474d-b1f1-aed34bd55802 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:31.794674 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9780dd-41d6-474d-b1f1-aed34bd55802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:31.825752 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ee9780dd-41d6-474d-b1f1-aed34bd55802 None None] MFA Rules not processed for user `e8fc5c19560f4f5ea3cd1390e2c9bff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:31.827032 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ee9780dd-41d6-474d-b1f1-aed34bd55802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:31.896135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1259/2527] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:31.907369 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0046812-8ca9-4cb9-be76-5092e0c553e1 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:31.909214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f0046812-8ca9-4cb9-be76-5092e0c553e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:31.967196 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0046812-8ca9-4cb9-be76-5092e0c553e1 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:31.970864 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba68ae-b31f-4465-8434-6dacc4d22045 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f281d778f1404984b4d778a1c26aef1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:31.971495 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba68ae-b31f-4465-8434-6dacc4d22045 service glance] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:31.971776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba68ae-b31f-4465-8434-6dacc4d22045 service glance] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:31.972055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba68ae-b31f-4465-8434-6dacc4d22045 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:31.981294 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba68ae-b31f-4465-8434-6dacc4d22045 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:31.982727 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba68ae-b31f-4465-8434-6dacc4d22045 service glance] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:31.991875 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1269/2528] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:41.893160 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=64,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:41.987699 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:46.773006 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9780dd-41d6-474d-b1f1-aed34bd55802 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:46.778796 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ee9780dd-41d6-474d-b1f1-aed34bd55802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:46.861487 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9780dd-41d6-474d-b1f1-aed34bd55802 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:46.867793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45def9dc-b368-48d8-9f25-a297d933d79f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:46.868424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45def9dc-b368-48d8-9f25-a297d933d79f service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:46.868699 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45def9dc-b368-48d8-9f25-a297d933d79f service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:46.868974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45def9dc-b368-48d8-9f25-a297d933d79f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:46.870773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-45def9dc-b368-48d8-9f25-a297d933d79f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:46.885648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45def9dc-b368-48d8-9f25-a297d933d79f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:46.887296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45def9dc-b368-48d8-9f25-a297d933d79f service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:46.901010 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1260/2529] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:36:51.879282 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba68ae-b31f-4465-8434-6dacc4d22045 service glance] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:51.880913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-91ba68ae-b31f-4465-8434-6dacc4d22045 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:51.953099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba68ae-b31f-4465-8434-6dacc4d22045 service glance] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:51.958423 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604b9c0-2d1f-4723-bf41-45127693bcef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:51.963595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604b9c0-2d1f-4723-bf41-45127693bcef service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:51.963595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604b9c0-2d1f-4723-bf41-45127693bcef service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:51.963595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604b9c0-2d1f-4723-bf41-45127693bcef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:51.963595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c604b9c0-2d1f-4723-bf41-45127693bcef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:52.037391 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604b9c0-2d1f-4723-bf41-45127693bcef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:52.038863 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604b9c0-2d1f-4723-bf41-45127693bcef service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:52.049770 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1270/2530] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:56.517164 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45def9dc-b368-48d8-9f25-a297d933d79f service nova] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:56.518690 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-45def9dc-b368-48d8-9f25-a297d933d79f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:56.533074 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45def9dc-b368-48d8-9f25-a297d933d79f service nova] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:56.538075 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4748d1-19dd-4cec-bc51-20885436b8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:56.538714 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4748d1-19dd-4cec-bc51-20885436b8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:56.539099 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4748d1-19dd-4cec-bc51-20885436b8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:56.539381 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4748d1-19dd-4cec-bc51-20885436b8d7 admin admin] PATH_INFO: `/v3/users/0d5336bcea514f378e06737680ae9bdc` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:56.540540 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4748d1-19dd-4cec-bc51-20885436b8d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d5336bcea514f378e06737680ae9bdc)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:56.542479 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4748d1-19dd-4cec-bc51-20885436b8d7 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:56.783856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-bb4748d1-19dd-4cec-bc51-20885436b8d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d5336bcea514f378e06737680ae9bdc'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:56.845550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-bb4748d1-19dd-4cec-bc51-20885436b8d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d5336bcea514f378e06737680ae9bdc'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:56.849146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-bb4748d1-19dd-4cec-bc51-20885436b8d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d5336bcea514f378e06737680ae9bdc'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:56.864942 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-bb4748d1-19dd-4cec-bc51-20885436b8d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d5336bcea514f378e06737680ae9bdc'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:56.865988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-bb4748d1-19dd-4cec-bc51-20885436b8d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d5336bcea514f378e06737680ae9bdc'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:56.879627 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1261/2531] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:36:56 2022] DELETE /identity/v3/users/0d5336bcea514f378e06737680ae9bdc => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:57.147986 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604b9c0-2d1f-4723-bf41-45127693bcef service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:57.160394 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604b9c0-2d1f-4723-bf41-45127693bcef service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:57.237520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f31433-cf8d-4777-af9c-6907266b2d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:57.237520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f31433-cf8d-4777-af9c-6907266b2d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:57.237520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f31433-cf8d-4777-af9c-6907266b2d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:57.237520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f31433-cf8d-4777-af9c-6907266b2d5d admin admin] PATH_INFO: `/v3/projects/4b5588a49a314579a29bad0f73aae8b4` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:57.238054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f31433-cf8d-4777-af9c-6907266b2d5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b5588a49a314579a29bad0f73aae8b4)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:57.239139 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f31433-cf8d-4777-af9c-6907266b2d5d admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:57.344592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f9f31433-cf8d-4777-af9c-6907266b2d5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b5588a49a314579a29bad0f73aae8b4', 'request_id': 'req-f9f31433-cf8d-4777-af9c-6907266b2d5d'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:57.346548 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:57.350580 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f9f31433-cf8d-4777-af9c-6907266b2d5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b5588a49a314579a29bad0f73aae8b4', 'request_id': 'req-f9f31433-cf8d-4777-af9c-6907266b2d5d'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:57.422080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f9f31433-cf8d-4777-af9c-6907266b2d5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b5588a49a314579a29bad0f73aae8b4', 'request_id': 'req-f9f31433-cf8d-4777-af9c-6907266b2d5d'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:57.433849 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f9f31433-cf8d-4777-af9c-6907266b2d5d admin admin] The token cache is being invalidate because project 4b5588a49a314579a29bad0f73aae8b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:36:57.434140 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f9f31433-cf8d-4777-af9c-6907266b2d5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:57.434791 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-f9f31433-cf8d-4777-af9c-6907266b2d5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:57.437009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1271/2532] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:36:57 2022] DELETE /identity/v3/projects/4b5588a49a314579a29bad0f73aae8b4 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:57.442022 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ca159-7496-4d5f-aced-223ff583c4f5 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:57.442293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ca159-7496-4d5f-aced-223ff583c4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:57.442572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ca159-7496-4d5f-aced-223ff583c4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:57.464519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-d34ca159-7496-4d5f-aced-223ff583c4f5 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:57.465726 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-d34ca159-7496-4d5f-aced-223ff583c4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:57.530166 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:57.541824 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1262/2533] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:57.550055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d59c79-817a-4d35-8071-3207f024fd42 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:57.550326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d59c79-817a-4d35-8071-3207f024fd42 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:57.550608 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d59c79-817a-4d35-8071-3207f024fd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:57.573978 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d5d59c79-817a-4d35-8071-3207f024fd42 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:57.575283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d5d59c79-817a-4d35-8071-3207f024fd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:57.585235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1272/2534] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:57.589538 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ca159-7496-4d5f-aced-223ff583c4f5 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:57.600323 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ca159-7496-4d5f-aced-223ff583c4f5 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:57.605542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcb2de-8645-42da-8ba6-933708c639c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:57.606735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcb2de-8645-42da-8ba6-933708c639c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:57.607143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcb2de-8645-42da-8ba6-933708c639c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:57.607643 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcb2de-8645-42da-8ba6-933708c639c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:57.608338 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bcb2de-8645-42da-8ba6-933708c639c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:36:57.608653 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bcb2de-8645-42da-8ba6-933708c639c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:57.610312 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bcb2de-8645-42da-8ba6-933708c639c8 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:57.622344 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1263/2535] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:36:57.627598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d59c79-817a-4d35-8071-3207f024fd42 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:57.637126 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d59c79-817a-4d35-8071-3207f024fd42 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:57.642571 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba4a91-0a98-44cf-86f9-e8ccd3a0bd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:57.645235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba4a91-0a98-44cf-86f9-e8ccd3a0bd2f admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:57.645235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba4a91-0a98-44cf-86f9-e8ccd3a0bd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:57.645235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba4a91-0a98-44cf-86f9-e8ccd3a0bd2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:57.645235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba4a91-0a98-44cf-86f9-e8ccd3a0bd2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:57.646262 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba4a91-0a98-44cf-86f9-e8ccd3a0bd2f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:57.721561 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1273/2536] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:36:57 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:57.726453 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcb2de-8645-42da-8ba6-933708c639c8 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:57.737390 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcb2de-8645-42da-8ba6-933708c639c8 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:57.793490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388cb5c-302e-4197-8ace-a9a1c8e8a314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:57.794102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388cb5c-302e-4197-8ace-a9a1c8e8a314 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:57.795049 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388cb5c-302e-4197-8ace-a9a1c8e8a314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:57.795330 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388cb5c-302e-4197-8ace-a9a1c8e8a314 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:57.795971 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388cb5c-302e-4197-8ace-a9a1c8e8a314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:57.797853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388cb5c-302e-4197-8ace-a9a1c8e8a314 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:57.891639 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1264/2537] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:36:57 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:36:57.896053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba4a91-0a98-44cf-86f9-e8ccd3a0bd2f admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:57.907198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba4a91-0a98-44cf-86f9-e8ccd3a0bd2f admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:57.912067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03a2d6-894d-4433-93c7-7da8091bd60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:57.912758 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03a2d6-894d-4433-93c7-7da8091bd60a admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:57.913037 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03a2d6-894d-4433-93c7-7da8091bd60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:57.913313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03a2d6-894d-4433-93c7-7da8091bd60a admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:57.913954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c03a2d6-894d-4433-93c7-7da8091bd60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:57.915643 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c03a2d6-894d-4433-93c7-7da8091bd60a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:57.926318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1274/2538] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:57.930281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388cb5c-302e-4197-8ace-a9a1c8e8a314 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:57.941236 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388cb5c-302e-4197-8ace-a9a1c8e8a314 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:57.946092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085ac79-73e2-4cf7-8f5a-ef9924167ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:57.946701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085ac79-73e2-4cf7-8f5a-ef9924167ecc admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:57.947025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085ac79-73e2-4cf7-8f5a-ef9924167ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:57.947303 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085ac79-73e2-4cf7-8f5a-ef9924167ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:57.947944 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d085ac79-73e2-4cf7-8f5a-ef9924167ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:57.949526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d085ac79-73e2-4cf7-8f5a-ef9924167ecc admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:57.960786 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1265/2539] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:57.965054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03a2d6-894d-4433-93c7-7da8091bd60a admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:57.975108 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03a2d6-894d-4433-93c7-7da8091bd60a admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:57.979732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40736929-3536-401f-8ad2-c8e09f961f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:57.980378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40736929-3536-401f-8ad2-c8e09f961f8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:57.980659 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40736929-3536-401f-8ad2-c8e09f961f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:57.980932 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40736929-3536-401f-8ad2-c8e09f961f8b admin admin] PATH_INFO: `/v3/projects/81de45a633e64b958a13f991d1a4bf9f/users/109c80ca49cd41b49330776bebf4c6a0/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:57.996250 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40736929-3536-401f-8ad2-c8e09f961f8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=81de45a633e64b958a13f991d1a4bf9f, user_id=109c80ca49cd41b49330776bebf4c6a0, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:58.000037 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40736929-3536-401f-8ad2-c8e09f961f8b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:58.057001 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1275/2540] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:36:57 2022] PUT /identity/v3/projects/81de45a633e64b958a13f991d1a4bf9f/users/109c80ca49cd41b49330776bebf4c6a0/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:58.068532 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582eab2-d2e1-45af-92e6-dfea16348bb0 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:58.069107 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582eab2-d2e1-45af-92e6-dfea16348bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:58.069457 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582eab2-d2e1-45af-92e6-dfea16348bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:58.098841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-1582eab2-d2e1-45af-92e6-dfea16348bb0 None None] MFA Rules not processed for user `109c80ca49cd41b49330776bebf4c6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:58.100105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-1582eab2-d2e1-45af-92e6-dfea16348bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:58.165336 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1266/2541] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:58.177962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40736929-3536-401f-8ad2-c8e09f961f8b admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:58.179679 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-40736929-3536-401f-8ad2-c8e09f961f8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:58.264032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40736929-3536-401f-8ad2-c8e09f961f8b admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:58.270218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa824e-bba6-4d1c-9216-963d3fa23ee1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:58.270840 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa824e-bba6-4d1c-9216-963d3fa23ee1 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:58.271159 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa824e-bba6-4d1c-9216-963d3fa23ee1 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:58.271436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa824e-bba6-4d1c-9216-963d3fa23ee1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:58.282414 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa824e-bba6-4d1c-9216-963d3fa23ee1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:58.283884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa824e-bba6-4d1c-9216-963d3fa23ee1 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:58.294994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1276/2542] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:58.520966 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582eab2-d2e1-45af-92e6-dfea16348bb0 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:58.580242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:58.595745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582eab2-d2e1-45af-92e6-dfea16348bb0 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:58.600465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222abf4-0582-454a-a715-3dd88f0ddba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:58.601114 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222abf4-0582-454a-a715-3dd88f0ddba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:58.601401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222abf4-0582-454a-a715-3dd88f0ddba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:58.601679 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222abf4-0582-454a-a715-3dd88f0ddba8 admin admin] PATH_INFO: `/v3/users/109c80ca49cd41b49330776bebf4c6a0` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:58.603546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222abf4-0582-454a-a715-3dd88f0ddba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=109c80ca49cd41b49330776bebf4c6a0)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:58.605471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222abf4-0582-454a-a715-3dd88f0ddba8 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:58.801919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7222abf4-0582-454a-a715-3dd88f0ddba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '109c80ca49cd41b49330776bebf4c6a0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:58.902585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7222abf4-0582-454a-a715-3dd88f0ddba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '109c80ca49cd41b49330776bebf4c6a0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:58.903278 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7222abf4-0582-454a-a715-3dd88f0ddba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '109c80ca49cd41b49330776bebf4c6a0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:58.916132 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7222abf4-0582-454a-a715-3dd88f0ddba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '109c80ca49cd41b49330776bebf4c6a0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:58.916806 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7222abf4-0582-454a-a715-3dd88f0ddba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '109c80ca49cd41b49330776bebf4c6a0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:58.929444 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1267/2543] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:36:58 2022] DELETE /identity/v3/users/109c80ca49cd41b49330776bebf4c6a0 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:58.933968 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f07f95-afc9-4c21-aca8-08bf11e17d53 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:58.937292 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f07f95-afc9-4c21-aca8-08bf11e17d53 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:58.937292 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f07f95-afc9-4c21-aca8-08bf11e17d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:58.955380 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d8f07f95-afc9-4c21-aca8-08bf11e17d53 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:58.960449 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d8f07f95-afc9-4c21-aca8-08bf11e17d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:59.017933 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1277/2544] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:36:59.027826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222abf4-0582-454a-a715-3dd88f0ddba8 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:59.029823 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-7222abf4-0582-454a-a715-3dd88f0ddba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:59.088507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222abf4-0582-454a-a715-3dd88f0ddba8 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:59.092826 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be7b32-c987-40d4-b754-b46718d04f2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:59.093364 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be7b32-c987-40d4-b754-b46718d04f2b service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:59.093643 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be7b32-c987-40d4-b754-b46718d04f2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:59.093914 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be7b32-c987-40d4-b754-b46718d04f2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:59.103633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be7b32-c987-40d4-b754-b46718d04f2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:59.105096 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be7b32-c987-40d4-b754-b46718d04f2b service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:59.116138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1268/2545] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:59.550613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f07f95-afc9-4c21-aca8-08bf11e17d53 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:59.552263 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d8f07f95-afc9-4c21-aca8-08bf11e17d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:59.559478 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:59.566212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f07f95-afc9-4c21-aca8-08bf11e17d53 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:59.574444 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de6b9d-37d4-45cd-afb4-6d4d1c4797d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:59.575535 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de6b9d-37d4-45cd-afb4-6d4d1c4797d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:59.576190 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de6b9d-37d4-45cd-afb4-6d4d1c4797d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:59.576449 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de6b9d-37d4-45cd-afb4-6d4d1c4797d7 admin admin] PATH_INFO: `/v3/projects/81de45a633e64b958a13f991d1a4bf9f` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:59.577841 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de6b9d-37d4-45cd-afb4-6d4d1c4797d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81de45a633e64b958a13f991d1a4bf9f)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:59.580306 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de6b9d-37d4-45cd-afb4-6d4d1c4797d7 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:59.704849 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d6de6b9d-37d4-45cd-afb4-6d4d1c4797d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81de45a633e64b958a13f991d1a4bf9f', 'request_id': 'req-d6de6b9d-37d4-45cd-afb4-6d4d1c4797d7'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:59.734850 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d6de6b9d-37d4-45cd-afb4-6d4d1c4797d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81de45a633e64b958a13f991d1a4bf9f', 'request_id': 'req-d6de6b9d-37d4-45cd-afb4-6d4d1c4797d7'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:59.821646 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d6de6b9d-37d4-45cd-afb4-6d4d1c4797d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81de45a633e64b958a13f991d1a4bf9f', 'request_id': 'req-d6de6b9d-37d4-45cd-afb4-6d4d1c4797d7'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:59.835830 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d6de6b9d-37d4-45cd-afb4-6d4d1c4797d7 admin admin] The token cache is being invalidate because project 81de45a633e64b958a13f991d1a4bf9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:36:59.836229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d6de6b9d-37d4-45cd-afb4-6d4d1c4797d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:59.836952 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d6de6b9d-37d4-45cd-afb4-6d4d1c4797d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:59.838600 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1278/2546] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:36:59 2022] DELETE /identity/v3/projects/81de45a633e64b958a13f991d1a4bf9f => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:59.843974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49980d8-2aec-4c57-a364-271c3bd48492 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:59.844463 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49980d8-2aec-4c57-a364-271c3bd48492 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:59.844562 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49980d8-2aec-4c57-a364-271c3bd48492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:59.865277 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-f49980d8-2aec-4c57-a364-271c3bd48492 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:59.866462 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f49980d8-2aec-4c57-a364-271c3bd48492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:59.932456 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1269/2547] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:36:59.941158 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157948b-2209-4643-815b-b6324ca796c0 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:59.941445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157948b-2209-4643-815b-b6324ca796c0 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:59.941728 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157948b-2209-4643-815b-b6324ca796c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:59.961843 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-9157948b-2209-4643-815b-b6324ca796c0 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:59.963101 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9157948b-2209-4643-815b-b6324ca796c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:59.971733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1279/2548] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:59.975892 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49980d8-2aec-4c57-a364-271c3bd48492 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:59.986778 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49980d8-2aec-4c57-a364-271c3bd48492 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:59.991415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60754bb1-9719-4c3f-a0b8-f15544fd5a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:59.992026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60754bb1-9719-4c3f-a0b8-f15544fd5a14 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:59.992312 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60754bb1-9719-4c3f-a0b8-f15544fd5a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:59.992640 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60754bb1-9719-4c3f-a0b8-f15544fd5a14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:59.993267 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60754bb1-9719-4c3f-a0b8-f15544fd5a14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:36:59.993576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60754bb1-9719-4c3f-a0b8-f15544fd5a14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:59.995406 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60754bb1-9719-4c3f-a0b8-f15544fd5a14 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:00.009091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1270/2549] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:37:00.013552 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157948b-2209-4643-815b-b6324ca796c0 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:00.023287 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157948b-2209-4643-815b-b6324ca796c0 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:00.028305 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3e3cd-6446-4fff-a42d-091317636ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:00.029520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3e3cd-6446-4fff-a42d-091317636ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:00.029520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3e3cd-6446-4fff-a42d-091317636ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:00.029520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3e3cd-6446-4fff-a42d-091317636ad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:00.030165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3e3cd-6446-4fff-a42d-091317636ad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:00.032127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3e3cd-6446-4fff-a42d-091317636ad6 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:00.110253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1280/2550] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:37:00 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:37:00.115649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60754bb1-9719-4c3f-a0b8-f15544fd5a14 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:00.121182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:37:00.127216 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60754bb1-9719-4c3f-a0b8-f15544fd5a14 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:00.183878 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a000fb9-4e4e-42ea-aea4-c7c59643c08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:00.184620 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a000fb9-4e4e-42ea-aea4-c7c59643c08f admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:00.185593 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a000fb9-4e4e-42ea-aea4-c7c59643c08f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:00.185878 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a000fb9-4e4e-42ea-aea4-c7c59643c08f admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:00.186516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a000fb9-4e4e-42ea-aea4-c7c59643c08f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:00.188466 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a000fb9-4e4e-42ea-aea4-c7c59643c08f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:00.291982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1271/2551] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:37:00 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:37:00.296672 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3e3cd-6446-4fff-a42d-091317636ad6 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:00.307063 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3e3cd-6446-4fff-a42d-091317636ad6 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:00.312011 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d2d3e-e588-4ba7-bd6a-8bfb10de3cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:00.312613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d2d3e-e588-4ba7-bd6a-8bfb10de3cfa admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:00.312896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d2d3e-e588-4ba7-bd6a-8bfb10de3cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:00.313170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d2d3e-e588-4ba7-bd6a-8bfb10de3cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:00.313843 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d2d3e-e588-4ba7-bd6a-8bfb10de3cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:00.315479 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d2d3e-e588-4ba7-bd6a-8bfb10de3cfa admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:00.325532 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1281/2552] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:37:00.329604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a000fb9-4e4e-42ea-aea4-c7c59643c08f admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:00.339917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a000fb9-4e4e-42ea-aea4-c7c59643c08f admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:00.345155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee9da7-6e93-4b16-b296-12e73288eab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:00.345760 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee9da7-6e93-4b16-b296-12e73288eab7 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:00.346045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee9da7-6e93-4b16-b296-12e73288eab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:00.346322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee9da7-6e93-4b16-b296-12e73288eab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:00.347048 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee9da7-6e93-4b16-b296-12e73288eab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:00.348643 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee9da7-6e93-4b16-b296-12e73288eab7 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:00.360433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1272/2553] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:37:00.365009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d2d3e-e588-4ba7-bd6a-8bfb10de3cfa admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:00.375613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d2d3e-e588-4ba7-bd6a-8bfb10de3cfa admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:00.380232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9b09e-73cc-4822-acbf-89e064a2b9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:00.385494 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9b09e-73cc-4822-acbf-89e064a2b9a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:00.385494 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9b09e-73cc-4822-acbf-89e064a2b9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:00.385494 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9b09e-73cc-4822-acbf-89e064a2b9a2 admin admin] PATH_INFO: `/v3/projects/5ead4251bd584e50b4a0d4898f14c795/users/d9317204628948aa9904dec57df3dfb4/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:00.397501 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9b09e-73cc-4822-acbf-89e064a2b9a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ead4251bd584e50b4a0d4898f14c795, user_id=d9317204628948aa9904dec57df3dfb4, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:00.400235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9b09e-73cc-4822-acbf-89e064a2b9a2 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:00.497040 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1282/2554] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:37:00 2022] PUT /identity/v3/projects/5ead4251bd584e50b4a0d4898f14c795/users/d9317204628948aa9904dec57df3dfb4/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:37:00.509471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8b09f-8ebc-4d5e-a26e-066bd0443c80 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:00.509756 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8b09f-8ebc-4d5e-a26e-066bd0443c80 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:00.510032 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8b09f-8ebc-4d5e-a26e-066bd0443c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:00.538715 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-f6c8b09f-8ebc-4d5e-a26e-066bd0443c80 None None] MFA Rules not processed for user `d9317204628948aa9904dec57df3dfb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:37:00.540109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f6c8b09f-8ebc-4d5e-a26e-066bd0443c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:00.606375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1273/2555] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:37:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:37:00.611903 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9b09e-73cc-4822-acbf-89e064a2b9a2 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:00.675761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9b09e-73cc-4822-acbf-89e064a2b9a2 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:00.680659 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed7815-dae1-4d6c-a35d-764dc8de10fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:00.681268 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed7815-dae1-4d6c-a35d-764dc8de10fe admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:00.681544 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed7815-dae1-4d6c-a35d-764dc8de10fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:00.681812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed7815-dae1-4d6c-a35d-764dc8de10fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:00.682448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed7815-dae1-4d6c-a35d-764dc8de10fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:00.684356 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed7815-dae1-4d6c-a35d-764dc8de10fe admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:00.795213 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1283/2556] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:37:00 2022] POST /identity/v3/projects => generated 371 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:37:00.799610 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8b09f-8ebc-4d5e-a26e-066bd0443c80 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:00.810471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8b09f-8ebc-4d5e-a26e-066bd0443c80 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:00.870642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74219dc-3ed2-49b7-b55f-77505a920234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:00.871568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74219dc-3ed2-49b7-b55f-77505a920234 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:00.871739 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74219dc-3ed2-49b7-b55f-77505a920234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:00.871885 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74219dc-3ed2-49b7-b55f-77505a920234 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:00.872559 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74219dc-3ed2-49b7-b55f-77505a920234 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:00.874613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74219dc-3ed2-49b7-b55f-77505a920234 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:00.966785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:37:00.972229 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1274/2557] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:37:00 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:37:00.976578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed7815-dae1-4d6c-a35d-764dc8de10fe admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:00.987129 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed7815-dae1-4d6c-a35d-764dc8de10fe admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:00.991897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d193b68-feae-4f33-97e4-ea09206313fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:00.992520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d193b68-feae-4f33-97e4-ea09206313fa admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:00.992800 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d193b68-feae-4f33-97e4-ea09206313fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:00.993072 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d193b68-feae-4f33-97e4-ea09206313fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:00.993709 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d193b68-feae-4f33-97e4-ea09206313fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:00.995381 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d193b68-feae-4f33-97e4-ea09206313fa admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:00.996692 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:37:01.006926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1284/2558] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:37:01.011152 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74219dc-3ed2-49b7-b55f-77505a920234 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:01.022045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74219dc-3ed2-49b7-b55f-77505a920234 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:01.026899 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da106bd-e6cf-4280-a160-4dfdae165c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:01.027571 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da106bd-e6cf-4280-a160-4dfdae165c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:01.027854 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da106bd-e6cf-4280-a160-4dfdae165c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:01.028131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da106bd-e6cf-4280-a160-4dfdae165c8e admin admin] PATH_INFO: `/v3/projects/00b6edd7fbd1495a974ee7dce40c3edd/users/8513cb977a0d4317a9ea28a9f5a45cbc/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:01.044413 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da106bd-e6cf-4280-a160-4dfdae165c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=00b6edd7fbd1495a974ee7dce40c3edd, user_id=8513cb977a0d4317a9ea28a9f5a45cbc, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:01.047109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da106bd-e6cf-4280-a160-4dfdae165c8e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:01.064086 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d193b68-feae-4f33-97e4-ea09206313fa admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:01.065885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9d193b68-feae-4f33-97e4-ea09206313fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:01.079415 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d193b68-feae-4f33-97e4-ea09206313fa admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:01.084372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca1968-d8ab-4af9-8d17-2aa25bf99cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:01.084996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca1968-d8ab-4af9-8d17-2aa25bf99cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:01.085269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca1968-d8ab-4af9-8d17-2aa25bf99cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:01.085535 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca1968-d8ab-4af9-8d17-2aa25bf99cf6 admin admin] PATH_INFO: `/v3/users/e8fc5c19560f4f5ea3cd1390e2c9bff5` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:01.086643 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca1968-d8ab-4af9-8d17-2aa25bf99cf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8fc5c19560f4f5ea3cd1390e2c9bff5)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:01.088591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca1968-d8ab-4af9-8d17-2aa25bf99cf6 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:01.114286 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1275/2559] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:37:01 2022] PUT /identity/v3/projects/00b6edd7fbd1495a974ee7dce40c3edd/users/8513cb977a0d4317a9ea28a9f5a45cbc/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:37:01.118395 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da106bd-e6cf-4280-a160-4dfdae165c8e admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:01.185543 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da106bd-e6cf-4280-a160-4dfdae165c8e admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:01.190450 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20364ee9-cade-49af-bf9a-beff50614892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:01.191171 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20364ee9-cade-49af-bf9a-beff50614892 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:01.191586 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20364ee9-cade-49af-bf9a-beff50614892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:01.191886 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20364ee9-cade-49af-bf9a-beff50614892 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:01.192525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20364ee9-cade-49af-bf9a-beff50614892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:01.194141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20364ee9-cade-49af-bf9a-beff50614892 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:01.205936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1276/2560] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:37:01.210103 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20364ee9-cade-49af-bf9a-beff50614892 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:01.220434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20364ee9-cade-49af-bf9a-beff50614892 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:01.225367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8a782-ff16-432b-b97b-2a92eb58cdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:01.226328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8a782-ff16-432b-b97b-2a92eb58cdab admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:01.226604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8a782-ff16-432b-b97b-2a92eb58cdab admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:01.226900 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8a782-ff16-432b-b97b-2a92eb58cdab admin admin] PATH_INFO: `/v3/projects/00b6edd7fbd1495a974ee7dce40c3edd/users/8513cb977a0d4317a9ea28a9f5a45cbc/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:01.228978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8a782-ff16-432b-b97b-2a92eb58cdab admin admin] RBAC: Authorizing `identity:create_grant(project_id=00b6edd7fbd1495a974ee7dce40c3edd, user_id=8513cb977a0d4317a9ea28a9f5a45cbc, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:01.232000 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8a782-ff16-432b-b97b-2a92eb58cdab admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:01.243396 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1277/2561] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:37:01 2022] PUT /identity/v3/projects/00b6edd7fbd1495a974ee7dce40c3edd/users/8513cb977a0d4317a9ea28a9f5a45cbc/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:37:01.254672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e167935-4ed3-45a2-88ba-0d9c56c9759f None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:01.254999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e167935-4ed3-45a2-88ba-0d9c56c9759f None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:01.255280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e167935-4ed3-45a2-88ba-0d9c56c9759f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:01.294404 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-5e167935-4ed3-45a2-88ba-0d9c56c9759f None None] MFA Rules not processed for user `8513cb977a0d4317a9ea28a9f5a45cbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:37:01.295916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-5e167935-4ed3-45a2-88ba-0d9c56c9759f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:01.324792 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-62ca1968-d8ab-4af9-8d17-2aa25bf99cf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8fc5c19560f4f5ea3cd1390e2c9bff5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:01.342121 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-62ca1968-d8ab-4af9-8d17-2aa25bf99cf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8fc5c19560f4f5ea3cd1390e2c9bff5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:01.376667 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1278/2562] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:37:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:37:01.391298 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e167935-4ed3-45a2-88ba-0d9c56c9759f None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:01.393089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-5e167935-4ed3-45a2-88ba-0d9c56c9759f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:01.400512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-62ca1968-d8ab-4af9-8d17-2aa25bf99cf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8fc5c19560f4f5ea3cd1390e2c9bff5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:01.410496 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-62ca1968-d8ab-4af9-8d17-2aa25bf99cf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8fc5c19560f4f5ea3cd1390e2c9bff5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:01.411158 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-62ca1968-d8ab-4af9-8d17-2aa25bf99cf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8fc5c19560f4f5ea3cd1390e2c9bff5'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:01.421804 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1285/2563] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:37:01 2022] DELETE /identity/v3/users/e8fc5c19560f4f5ea3cd1390e2c9bff5 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:37:01.423398 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e440e-94ab-4779-9bd8-150a3bd86c94 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:01.423768 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e440e-94ab-4779-9bd8-150a3bd86c94 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:01.424096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e440e-94ab-4779-9bd8-150a3bd86c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:01.446643 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-3a2e440e-94ab-4779-9bd8-150a3bd86c94 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:37:01.447849 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3a2e440e-94ab-4779-9bd8-150a3bd86c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:01.449497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:37:01.461917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e167935-4ed3-45a2-88ba-0d9c56c9759f None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:01.519874 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1286/2564] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:37:01.532614 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e440e-94ab-4779-9bd8-150a3bd86c94 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:01.534315 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3a2e440e-94ab-4779-9bd8-150a3bd86c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:01.536380 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2979a0b-e2a8-4d22-b820-247dabfef6a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:01.537066 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2979a0b-e2a8-4d22-b820-247dabfef6a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:01.537402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2979a0b-e2a8-4d22-b820-247dabfef6a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:01.537744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2979a0b-e2a8-4d22-b820-247dabfef6a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:01.539595 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e2979a0b-e2a8-4d22-b820-247dabfef6a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:01.603983 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e440e-94ab-4779-9bd8-150a3bd86c94 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:01.608527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1c018-9f85-4e0d-a5ba-b823aadc2754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:01.609244 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1c018-9f85-4e0d-a5ba-b823aadc2754 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:01.609523 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1c018-9f85-4e0d-a5ba-b823aadc2754 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:01.609907 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1c018-9f85-4e0d-a5ba-b823aadc2754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:01.618291 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2979a0b-e2a8-4d22-b820-247dabfef6a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:01.619773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2979a0b-e2a8-4d22-b820-247dabfef6a4 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:01.619932 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1c018-9f85-4e0d-a5ba-b823aadc2754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:01.621360 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1c018-9f85-4e0d-a5ba-b823aadc2754 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:01.631058 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1279/2565] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:37:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:37:01.632190 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1287/2566] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:37:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:37:01.909952 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2979a0b-e2a8-4d22-b820-247dabfef6a4 service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:01.911638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e2979a0b-e2a8-4d22-b820-247dabfef6a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:01.923576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2979a0b-e2a8-4d22-b820-247dabfef6a4 service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:01.927338 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bec9e-319b-47e7-9503-2acf8f694b2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:01.927960 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bec9e-319b-47e7-9503-2acf8f694b2b service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:01.928247 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bec9e-319b-47e7-9503-2acf8f694b2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:01.928524 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bec9e-319b-47e7-9503-2acf8f694b2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:01.930354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a24bec9e-319b-47e7-9503-2acf8f694b2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:02.003104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24bec9e-319b-47e7-9503-2acf8f694b2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:02.004520 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24bec9e-319b-47e7-9503-2acf8f694b2b service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:02.014379 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1280/2567] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:37:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:37:02.081559 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1c018-9f85-4e0d-a5ba-b823aadc2754 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:02.083359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-47e1c018-9f85-4e0d-a5ba-b823aadc2754 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:02.105679 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1c018-9f85-4e0d-a5ba-b823aadc2754 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:02.112436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577040a2-4ee0-41d3-ad22-35a8f04fbdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:02.115848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577040a2-4ee0-41d3-ad22-35a8f04fbdda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:02.115848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577040a2-4ee0-41d3-ad22-35a8f04fbdda admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:02.115848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577040a2-4ee0-41d3-ad22-35a8f04fbdda admin admin] PATH_INFO: `/v3/projects/fbccb26141674211a27732b014667ed7` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:02.115848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577040a2-4ee0-41d3-ad22-35a8f04fbdda admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbccb26141674211a27732b014667ed7)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:02.117458 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577040a2-4ee0-41d3-ad22-35a8f04fbdda admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:02.124793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:37:02.306024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-577040a2-4ee0-41d3-ad22-35a8f04fbdda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbccb26141674211a27732b014667ed7', 'request_id': 'req-577040a2-4ee0-41d3-ad22-35a8f04fbdda'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:02.312625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-577040a2-4ee0-41d3-ad22-35a8f04fbdda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbccb26141674211a27732b014667ed7', 'request_id': 'req-577040a2-4ee0-41d3-ad22-35a8f04fbdda'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:02.363550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-577040a2-4ee0-41d3-ad22-35a8f04fbdda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbccb26141674211a27732b014667ed7', 'request_id': 'req-577040a2-4ee0-41d3-ad22-35a8f04fbdda'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:02.386547 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-577040a2-4ee0-41d3-ad22-35a8f04fbdda admin admin] The token cache is being invalidate because project fbccb26141674211a27732b014667ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:37:02.386937 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-577040a2-4ee0-41d3-ad22-35a8f04fbdda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:02.387619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-577040a2-4ee0-41d3-ad22-35a8f04fbdda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:02.390402 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1288/2568] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:37:02 2022] DELETE /identity/v3/projects/fbccb26141674211a27732b014667ed7 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:37:02.395704 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498ad11-bb25-424d-8cab-fcf07134daef None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:02.396367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498ad11-bb25-424d-8cab-fcf07134daef None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:02.397001 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498ad11-bb25-424d-8cab-fcf07134daef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:02.426093 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-8498ad11-bb25-424d-8cab-fcf07134daef None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:37:02.427858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8498ad11-bb25-424d-8cab-fcf07134daef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:02.514084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:37:02.543050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1281/2569] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:37:02.554055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd49e4-acdd-481d-a4f8-4e7f37efe66e None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:02.554364 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd49e4-acdd-481d-a4f8-4e7f37efe66e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:02.554659 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd49e4-acdd-481d-a4f8-4e7f37efe66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:02.593160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-27bd49e4-acdd-481d-a4f8-4e7f37efe66e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:37:02.594398 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-27bd49e4-acdd-481d-a4f8-4e7f37efe66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:02.604813 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1289/2570] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:37:02.609616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498ad11-bb25-424d-8cab-fcf07134daef None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:02.619657 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498ad11-bb25-424d-8cab-fcf07134daef None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:02.626084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faa6d6-771f-4a4e-bb89-f8fca61d0e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:02.627898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faa6d6-771f-4a4e-bb89-f8fca61d0e90 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:02.628276 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faa6d6-771f-4a4e-bb89-f8fca61d0e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:02.628467 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faa6d6-771f-4a4e-bb89-f8fca61d0e90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:02.629139 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faa6d6-771f-4a4e-bb89-f8fca61d0e90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:37:02.629447 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faa6d6-771f-4a4e-bb89-f8fca61d0e90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:02.631239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faa6d6-771f-4a4e-bb89-f8fca61d0e90 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:02.651398 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1282/2571] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:37:02.657160 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd49e4-acdd-481d-a4f8-4e7f37efe66e None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:02.667091 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd49e4-acdd-481d-a4f8-4e7f37efe66e None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:02.672975 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209cc58b-d039-40fe-8d5d-19672467f8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:02.673594 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cc58b-d039-40fe-8d5d-19672467f8fd admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:02.674126 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cc58b-d039-40fe-8d5d-19672467f8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:02.674640 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cc58b-d039-40fe-8d5d-19672467f8fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:02.675571 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209cc58b-d039-40fe-8d5d-19672467f8fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:02.677591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209cc58b-d039-40fe-8d5d-19672467f8fd admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:02.786596 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1290/2572] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:37:02 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:37:02.788066 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faa6d6-771f-4a4e-bb89-f8fca61d0e90 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:02.799892 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faa6d6-771f-4a4e-bb89-f8fca61d0e90 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:02.875628 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a56fa6-eab9-4f05-b2ad-b531b6f67577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:02.876564 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a56fa6-eab9-4f05-b2ad-b531b6f67577 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:02.877209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a56fa6-eab9-4f05-b2ad-b531b6f67577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:02.877461 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a56fa6-eab9-4f05-b2ad-b531b6f67577 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:02.878288 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a56fa6-eab9-4f05-b2ad-b531b6f67577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:02.881584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a56fa6-eab9-4f05-b2ad-b531b6f67577 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:02.969729 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1283/2573] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:37:02 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:37:02.973942 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209cc58b-d039-40fe-8d5d-19672467f8fd admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:02.986542 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209cc58b-d039-40fe-8d5d-19672467f8fd admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:02.991749 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc5503-6fba-435a-bd9f-78995f1f90f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:02.992401 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc5503-6fba-435a-bd9f-78995f1f90f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:02.992688 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc5503-6fba-435a-bd9f-78995f1f90f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:02.992965 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc5503-6fba-435a-bd9f-78995f1f90f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:02.993622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc5503-6fba-435a-bd9f-78995f1f90f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:02.995330 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc5503-6fba-435a-bd9f-78995f1f90f8 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:03.006152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1291/2574] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:37:03.010265 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a56fa6-eab9-4f05-b2ad-b531b6f67577 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:03.021031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a56fa6-eab9-4f05-b2ad-b531b6f67577 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:03.025768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16a214-05f2-4237-b20f-29ca4f182554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:03.027014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16a214-05f2-4237-b20f-29ca4f182554 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:03.027014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16a214-05f2-4237-b20f-29ca4f182554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:03.027219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16a214-05f2-4237-b20f-29ca4f182554 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:03.027866 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16a214-05f2-4237-b20f-29ca4f182554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:03.030223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16a214-05f2-4237-b20f-29ca4f182554 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:03.042545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1284/2575] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:37:03.047167 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc5503-6fba-435a-bd9f-78995f1f90f8 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:03.057981 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc5503-6fba-435a-bd9f-78995f1f90f8 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:03.063737 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e607a2b-4619-429f-aa11-a4b668923116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:03.064418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e607a2b-4619-429f-aa11-a4b668923116 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:03.064700 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e607a2b-4619-429f-aa11-a4b668923116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:03.064977 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e607a2b-4619-429f-aa11-a4b668923116 admin admin] PATH_INFO: `/v3/projects/b8270db003fa4442afb38ba99ee0263e/users/1d68da36768d48a2bccb0fd2fd1a7acb/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:03.082122 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e607a2b-4619-429f-aa11-a4b668923116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8270db003fa4442afb38ba99ee0263e, user_id=1d68da36768d48a2bccb0fd2fd1a7acb, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:03.085338 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e607a2b-4619-429f-aa11-a4b668923116 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:03.172536 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1292/2576] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:37:03 2022] PUT /identity/v3/projects/b8270db003fa4442afb38ba99ee0263e/users/1d68da36768d48a2bccb0fd2fd1a7acb/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:37:03.177795 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439397a4-11ff-4493-b52d-45a5739c1f0b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:03.178245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439397a4-11ff-4493-b52d-45a5739c1f0b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:03.178438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439397a4-11ff-4493-b52d-45a5739c1f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:03.203954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-439397a4-11ff-4493-b52d-45a5739c1f0b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:37:03.205227 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-439397a4-11ff-4493-b52d-45a5739c1f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:03.269897 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1285/2577] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:37:03.280525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e607a2b-4619-429f-aa11-a4b668923116 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:03.282392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4e607a2b-4619-429f-aa11-a4b668923116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:03.341810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e607a2b-4619-429f-aa11-a4b668923116 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:03.345973 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdabc67-a4d0-48a4-8b33-566c740352cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:03.346649 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdabc67-a4d0-48a4-8b33-566c740352cf service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:03.346948 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdabc67-a4d0-48a4-8b33-566c740352cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:03.347227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdabc67-a4d0-48a4-8b33-566c740352cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:03.360278 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdabc67-a4d0-48a4-8b33-566c740352cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:03.362250 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdabc67-a4d0-48a4-8b33-566c740352cf service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:03.375286 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1293/2578] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:37:04.860853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf500b6-d078-4c71-86d8-243da920879c None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:04.860853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf500b6-d078-4c71-86d8-243da920879c None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:04.860853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf500b6-d078-4c71-86d8-243da920879c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:04.894783 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-dcf500b6-d078-4c71-86d8-243da920879c None None] MFA Rules not processed for user `1d68da36768d48a2bccb0fd2fd1a7acb`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:37:04.896008 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-dcf500b6-d078-4c71-86d8-243da920879c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:04.948196 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:37:04.975063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1286/2579] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:37:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:37:04.987736 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdabc67-a4d0-48a4-8b33-566c740352cf service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:04.989143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9cdabc67-a4d0-48a4-8b33-566c740352cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:05.035067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:37:05.060670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdabc67-a4d0-48a4-8b33-566c740352cf service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:05.065685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03006ca4-bf90-4ddd-be49-670a90255212 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:05.066366 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03006ca4-bf90-4ddd-be49-670a90255212 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:05.066649 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03006ca4-bf90-4ddd-be49-670a90255212 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:05.066957 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03006ca4-bf90-4ddd-be49-670a90255212 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:05.077358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03006ca4-bf90-4ddd-be49-670a90255212 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:05.078842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03006ca4-bf90-4ddd-be49-670a90255212 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:05.089355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1294/2580] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:37:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:37:08.091232 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0543275-f888-4bac-81af-25626980ec86 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:08.091883 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0543275-f888-4bac-81af-25626980ec86 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:08.091883 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0543275-f888-4bac-81af-25626980ec86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:08.118089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-c0543275-f888-4bac-81af-25626980ec86 None None] MFA Rules not processed for user `1d68da36768d48a2bccb0fd2fd1a7acb`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:37:08.119352 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c0543275-f888-4bac-81af-25626980ec86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:08.127557 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1287/2581] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:37:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:37:08.498189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03006ca4-bf90-4ddd-be49-670a90255212 service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:08.499669 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-03006ca4-bf90-4ddd-be49-670a90255212 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:08.512916 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03006ca4-bf90-4ddd-be49-670a90255212 service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:08.517048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba04165-5909-4e3e-a080-aae4f650a0c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:08.517818 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba04165-5909-4e3e-a080-aae4f650a0c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:08.518401 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba04165-5909-4e3e-a080-aae4f650a0c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:08.518711 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba04165-5909-4e3e-a080-aae4f650a0c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:08.520883 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-cba04165-5909-4e3e-a080-aae4f650a0c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:08.536091 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba04165-5909-4e3e-a080-aae4f650a0c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:08.537452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba04165-5909-4e3e-a080-aae4f650a0c4 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:08.548497 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1295/2582] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:37:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:37:08.818888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0543275-f888-4bac-81af-25626980ec86 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:08.820550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c0543275-f888-4bac-81af-25626980ec86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:08.904142 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0543275-f888-4bac-81af-25626980ec86 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:08.911799 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b897c3c4-92ef-48d0-aa60-43d89651a48c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:08.912555 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897c3c4-92ef-48d0-aa60-43d89651a48c service placement] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:08.912842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897c3c4-92ef-48d0-aa60-43d89651a48c service placement] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:08.913122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897c3c4-92ef-48d0-aa60-43d89651a48c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:08.920937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b897c3c4-92ef-48d0-aa60-43d89651a48c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:08.934603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b897c3c4-92ef-48d0-aa60-43d89651a48c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:08.936714 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b897c3c4-92ef-48d0-aa60-43d89651a48c service placement] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:08.947253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1288/2583] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:37:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:37:18.234847 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba04165-5909-4e3e-a080-aae4f650a0c4 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:18.236492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-cba04165-5909-4e3e-a080-aae4f650a0c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:18.249930 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba04165-5909-4e3e-a080-aae4f650a0c4 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:18.255084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfd798-12f5-4469-a0e5-6875f40eafa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:18.255764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfd798-12f5-4469-a0e5-6875f40eafa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:18.256137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfd798-12f5-4469-a0e5-6875f40eafa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:18.256418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfd798-12f5-4469-a0e5-6875f40eafa6 admin admin] PATH_INFO: `/v3/users/be55f0bb21f044b0a8346cf953b1650e` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:18.272233 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfd798-12f5-4469-a0e5-6875f40eafa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be55f0bb21f044b0a8346cf953b1650e)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:18.274523 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfd798-12f5-4469-a0e5-6875f40eafa6 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:18.482256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-aacfd798-12f5-4469-a0e5-6875f40eafa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be55f0bb21f044b0a8346cf953b1650e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:18.495622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-aacfd798-12f5-4469-a0e5-6875f40eafa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be55f0bb21f044b0a8346cf953b1650e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:18.582232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-aacfd798-12f5-4469-a0e5-6875f40eafa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be55f0bb21f044b0a8346cf953b1650e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:18.590683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-aacfd798-12f5-4469-a0e5-6875f40eafa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be55f0bb21f044b0a8346cf953b1650e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:18.591597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-aacfd798-12f5-4469-a0e5-6875f40eafa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be55f0bb21f044b0a8346cf953b1650e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:18.598107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1296/2584] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:37:18 2022] DELETE /identity/v3/users/be55f0bb21f044b0a8346cf953b1650e => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:37:18.859078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b897c3c4-92ef-48d0-aa60-43d89651a48c service placement] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:18.872579 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b897c3c4-92ef-48d0-aa60-43d89651a48c service placement] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:18.936391 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:18.937078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:18.937356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:18.937628 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f admin admin] PATH_INFO: `/v3/projects/b8bfa72ff61547e5a3097e5b40483f8a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:18.948615 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8bfa72ff61547e5a3097e5b40483f8a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:18.950507 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:19.048108 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8bfa72ff61547e5a3097e5b40483f8a', 'request_id': 'req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:19.053375 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8bfa72ff61547e5a3097e5b40483f8a', 'request_id': 'req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:19.055455 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:37:19.061548 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8bfa72ff61547e5a3097e5b40483f8a', 'request_id': 'req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:19.170992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f admin admin] The token cache is being invalidate because project b8bfa72ff61547e5a3097e5b40483f8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:37:19.171339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:19.171978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:19.173482 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1289/2585] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:37:18 2022] DELETE /identity/v3/projects/b8bfa72ff61547e5a3097e5b40483f8a => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:37:19.177736 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfd798-12f5-4469-a0e5-6875f40eafa6 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:19.179230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-aacfd798-12f5-4469-a0e5-6875f40eafa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:19.192792 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfd798-12f5-4469-a0e5-6875f40eafa6 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:19.251406 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0c157-5dd5-44a1-95da-091b12ba4baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:19.257236 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0c157-5dd5-44a1-95da-091b12ba4baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:19.257236 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0c157-5dd5-44a1-95da-091b12ba4baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:19.257236 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0c157-5dd5-44a1-95da-091b12ba4baf admin admin] PATH_INFO: `/v3/users/d7ac3fe714fb48459a3524d4a458de62` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:19.267181 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0c157-5dd5-44a1-95da-091b12ba4baf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7ac3fe714fb48459a3524d4a458de62)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:19.269145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0c157-5dd5-44a1-95da-091b12ba4baf admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:19.283541 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:37:19.510545 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-aca0c157-5dd5-44a1-95da-091b12ba4baf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7ac3fe714fb48459a3524d4a458de62'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:19.523918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-aca0c157-5dd5-44a1-95da-091b12ba4baf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7ac3fe714fb48459a3524d4a458de62'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:19.644896 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-aca0c157-5dd5-44a1-95da-091b12ba4baf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7ac3fe714fb48459a3524d4a458de62'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:19.654000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-aca0c157-5dd5-44a1-95da-091b12ba4baf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7ac3fe714fb48459a3524d4a458de62'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:19.655034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-aca0c157-5dd5-44a1-95da-091b12ba4baf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7ac3fe714fb48459a3524d4a458de62'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:19.674422 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1297/2586] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:37:19 2022] DELETE /identity/v3/users/d7ac3fe714fb48459a3524d4a458de62 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:37:19.939455 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:19.941452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:19.958410 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c9b66-c3c2-4f9b-b1a0-e9337cc6a53f admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:20.024194 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b1d94-5064-4b10-bce9-8507c33165ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:20.024770 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b1d94-5064-4b10-bce9-8507c33165ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:20.025054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b1d94-5064-4b10-bce9-8507c33165ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:20.025336 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b1d94-5064-4b10-bce9-8507c33165ed admin admin] PATH_INFO: `/v3/projects/f1d33e3eb5ee498f9422a177338b0f9d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:20.036959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b1d94-5064-4b10-bce9-8507c33165ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1d33e3eb5ee498f9422a177338b0f9d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:20.038937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b1d94-5064-4b10-bce9-8507c33165ed admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:20.189102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-179b1d94-5064-4b10-bce9-8507c33165ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1d33e3eb5ee498f9422a177338b0f9d', 'request_id': 'req-179b1d94-5064-4b10-bce9-8507c33165ed'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:20.194691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-179b1d94-5064-4b10-bce9-8507c33165ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1d33e3eb5ee498f9422a177338b0f9d', 'request_id': 'req-179b1d94-5064-4b10-bce9-8507c33165ed'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:20.201965 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-179b1d94-5064-4b10-bce9-8507c33165ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1d33e3eb5ee498f9422a177338b0f9d', 'request_id': 'req-179b1d94-5064-4b10-bce9-8507c33165ed'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:20.307397 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-179b1d94-5064-4b10-bce9-8507c33165ed admin admin] The token cache is being invalidate because project f1d33e3eb5ee498f9422a177338b0f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:37:20.307781 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-179b1d94-5064-4b10-bce9-8507c33165ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:20.308412 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-179b1d94-5064-4b10-bce9-8507c33165ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:37:20.310762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1290/2587] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:37:19 2022] DELETE /identity/v3/projects/f1d33e3eb5ee498f9422a177338b0f9d => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:37:22.202430 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0c157-5dd5-44a1-95da-091b12ba4baf admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:22.204018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-aca0c157-5dd5-44a1-95da-091b12ba4baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:22.282098 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0c157-5dd5-44a1-95da-091b12ba4baf admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:22.350273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a6b3b-d73e-4e76-998d-de00683b9360 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:22.350923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a6b3b-d73e-4e76-998d-de00683b9360 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:22.351218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a6b3b-d73e-4e76-998d-de00683b9360 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:22.351497 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a6b3b-d73e-4e76-998d-de00683b9360 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:22.353392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a40a6b3b-d73e-4e76-998d-de00683b9360 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:22.366526 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a6b3b-d73e-4e76-998d-de00683b9360 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:22.368174 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a6b3b-d73e-4e76-998d-de00683b9360 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:22.380855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1298/2588] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:37:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:37:30.303770 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=58,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:37:31.925996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c0956-fbed-46ac-a481-27d2cc49b931 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:31.926592 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c0956-fbed-46ac-a481-27d2cc49b931 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:31.926592 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c0956-fbed-46ac-a481-27d2cc49b931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:31.947014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-378c0956-fbed-46ac-a481-27d2cc49b931 None None] MFA Rules not processed for user `1d68da36768d48a2bccb0fd2fd1a7acb`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:37:31.948244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-378c0956-fbed-46ac-a481-27d2cc49b931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:32.010219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1291/2589] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:37:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:37:32.376190 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=83,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:37:42.001718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=25 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:01.882781 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a6b3b-d73e-4e76-998d-de00683b9360 service nova] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:01.884260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a40a6b3b-d73e-4e76-998d-de00683b9360 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:01.964928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a6b3b-d73e-4e76-998d-de00683b9360 service nova] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:01.969846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8241f8-95f9-46e3-9183-3595765192c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:01.970596 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8241f8-95f9-46e3-9183-3595765192c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:01.971011 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8241f8-95f9-46e3-9183-3595765192c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:01.971426 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8241f8-95f9-46e3-9183-3595765192c8 admin admin] PATH_INFO: `/v3/users/d9317204628948aa9904dec57df3dfb4` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:01.972869 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8241f8-95f9-46e3-9183-3595765192c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9317204628948aa9904dec57df3dfb4)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:01.974732 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8241f8-95f9-46e3-9183-3595765192c8 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:02.168466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ec8241f8-95f9-46e3-9183-3595765192c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9317204628948aa9904dec57df3dfb4'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:02.182027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ec8241f8-95f9-46e3-9183-3595765192c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9317204628948aa9904dec57df3dfb4'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:02.249372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ec8241f8-95f9-46e3-9183-3595765192c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9317204628948aa9904dec57df3dfb4'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:02.257670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ec8241f8-95f9-46e3-9183-3595765192c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9317204628948aa9904dec57df3dfb4'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:02.258326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ec8241f8-95f9-46e3-9183-3595765192c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9317204628948aa9904dec57df3dfb4'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:02.264410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1299/2590] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:38:01 2022] DELETE /identity/v3/users/d9317204628948aa9904dec57df3dfb4 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:02.269025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5acc9-8a43-467a-85a2-8ef4bfc1e3f4 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:02.269612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5acc9-8a43-467a-85a2-8ef4bfc1e3f4 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:02.269721 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5acc9-8a43-467a-85a2-8ef4bfc1e3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:02.290664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-34d5acc9-8a43-467a-85a2-8ef4bfc1e3f4 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:02.291905 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-34d5acc9-8a43-467a-85a2-8ef4bfc1e3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:02.354441 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1292/2591] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:02.365238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8241f8-95f9-46e3-9183-3595765192c8 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:02.366622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ec8241f8-95f9-46e3-9183-3595765192c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:02.424422 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8241f8-95f9-46e3-9183-3595765192c8 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:02.429297 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3fdf1-a7a3-4060-91d7-c816004aaee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:02.429908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3fdf1-a7a3-4060-91d7-c816004aaee5 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:02.430191 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3fdf1-a7a3-4060-91d7-c816004aaee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:02.430550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3fdf1-a7a3-4060-91d7-c816004aaee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:02.439863 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3fdf1-a7a3-4060-91d7-c816004aaee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:02.441397 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3fdf1-a7a3-4060-91d7-c816004aaee5 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:02.451231 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1300/2592] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:38:02.898137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5acc9-8a43-467a-85a2-8ef4bfc1e3f4 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:02.899581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-34d5acc9-8a43-467a-85a2-8ef4bfc1e3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:02.914350 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5acc9-8a43-467a-85a2-8ef4bfc1e3f4 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:02.919063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a338716-5010-4c93-9432-74a8e8b0ba0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:02.919703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a338716-5010-4c93-9432-74a8e8b0ba0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:02.920008 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a338716-5010-4c93-9432-74a8e8b0ba0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:02.920290 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a338716-5010-4c93-9432-74a8e8b0ba0a admin admin] PATH_INFO: `/v3/projects/5ead4251bd584e50b4a0d4898f14c795` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:02.921429 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a338716-5010-4c93-9432-74a8e8b0ba0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ead4251bd584e50b4a0d4898f14c795)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:02.923373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a338716-5010-4c93-9432-74a8e8b0ba0a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:03.045289 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0a338716-5010-4c93-9432-74a8e8b0ba0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ead4251bd584e50b4a0d4898f14c795', 'request_id': 'req-0a338716-5010-4c93-9432-74a8e8b0ba0a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:03.051836 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0a338716-5010-4c93-9432-74a8e8b0ba0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ead4251bd584e50b4a0d4898f14c795', 'request_id': 'req-0a338716-5010-4c93-9432-74a8e8b0ba0a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:03.057690 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0a338716-5010-4c93-9432-74a8e8b0ba0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ead4251bd584e50b4a0d4898f14c795', 'request_id': 'req-0a338716-5010-4c93-9432-74a8e8b0ba0a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:03.163584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0a338716-5010-4c93-9432-74a8e8b0ba0a admin admin] The token cache is being invalidate because project 5ead4251bd584e50b4a0d4898f14c795 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:38:03.166291 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0a338716-5010-4c93-9432-74a8e8b0ba0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:03.166291 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0a338716-5010-4c93-9432-74a8e8b0ba0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:03.166291 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1293/2593] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:38:02 2022] DELETE /identity/v3/projects/5ead4251bd584e50b4a0d4898f14c795 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:03.169891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3fdf1-a7a3-4060-91d7-c816004aaee5 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:03.179841 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3fdf1-a7a3-4060-91d7-c816004aaee5 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:03.199761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:03.239597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ef9e5-b139-4c3b-a460-e1b38ed36d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:03.240152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ef9e5-b139-4c3b-a460-e1b38ed36d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:03.240434 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ef9e5-b139-4c3b-a460-e1b38ed36d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:03.240709 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ef9e5-b139-4c3b-a460-e1b38ed36d6b admin admin] PATH_INFO: `/v3/users/8513cb977a0d4317a9ea28a9f5a45cbc` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:03.241848 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ef9e5-b139-4c3b-a460-e1b38ed36d6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8513cb977a0d4317a9ea28a9f5a45cbc)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:03.243851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ef9e5-b139-4c3b-a460-e1b38ed36d6b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:03.480568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ef9ef9e5-b139-4c3b-a460-e1b38ed36d6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8513cb977a0d4317a9ea28a9f5a45cbc'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:03.498242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ef9ef9e5-b139-4c3b-a460-e1b38ed36d6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8513cb977a0d4317a9ea28a9f5a45cbc'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:03.581072 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ef9ef9e5-b139-4c3b-a460-e1b38ed36d6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8513cb977a0d4317a9ea28a9f5a45cbc'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:03.588720 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ef9ef9e5-b139-4c3b-a460-e1b38ed36d6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8513cb977a0d4317a9ea28a9f5a45cbc'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:03.590024 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ef9ef9e5-b139-4c3b-a460-e1b38ed36d6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8513cb977a0d4317a9ea28a9f5a45cbc'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:03.595210 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1301/2594] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:38:03 2022] DELETE /identity/v3/users/8513cb977a0d4317a9ea28a9f5a45cbc => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:38:03.962009 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a338716-5010-4c93-9432-74a8e8b0ba0a admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:03.963517 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-0a338716-5010-4c93-9432-74a8e8b0ba0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:04.006209 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ef9e5-b139-4c3b-a460-e1b38ed36d6b admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:04.007677 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ef9ef9e5-b139-4c3b-a460-e1b38ed36d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:04.106104 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ef9e5-b139-4c3b-a460-e1b38ed36d6b admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:04.109183 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304fb8a-32bf-4464-9804-28912fc7364e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:04.109834 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304fb8a-32bf-4464-9804-28912fc7364e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:04.110111 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304fb8a-32bf-4464-9804-28912fc7364e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:04.110439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304fb8a-32bf-4464-9804-28912fc7364e admin admin] PATH_INFO: `/v3/projects/00b6edd7fbd1495a974ee7dce40c3edd` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:04.111674 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304fb8a-32bf-4464-9804-28912fc7364e admin admin] RBAC: Authorizing `identity:delete_project(project_id=00b6edd7fbd1495a974ee7dce40c3edd)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:04.114948 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304fb8a-32bf-4464-9804-28912fc7364e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:04.183137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a338716-5010-4c93-9432-74a8e8b0ba0a admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:04.202606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:04.204639 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:04.208035 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-0304fb8a-32bf-4464-9804-28912fc7364e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00b6edd7fbd1495a974ee7dce40c3edd', 'request_id': 'req-0304fb8a-32bf-4464-9804-28912fc7364e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:04.214334 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-0304fb8a-32bf-4464-9804-28912fc7364e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00b6edd7fbd1495a974ee7dce40c3edd', 'request_id': 'req-0304fb8a-32bf-4464-9804-28912fc7364e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:04.255609 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09097429-14be-4ef7-aa28-c4a65615b88d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:04.256274 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09097429-14be-4ef7-aa28-c4a65615b88d service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:04.256502 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09097429-14be-4ef7-aa28-c4a65615b88d service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:04.256775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09097429-14be-4ef7-aa28-c4a65615b88d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:04.258513 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-09097429-14be-4ef7-aa28-c4a65615b88d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:04.295592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-0304fb8a-32bf-4464-9804-28912fc7364e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00b6edd7fbd1495a974ee7dce40c3edd', 'request_id': 'req-0304fb8a-32bf-4464-9804-28912fc7364e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:04.308068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-0304fb8a-32bf-4464-9804-28912fc7364e admin admin] The token cache is being invalidate because project 00b6edd7fbd1495a974ee7dce40c3edd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:38:04.308355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-0304fb8a-32bf-4464-9804-28912fc7364e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:04.308993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-0304fb8a-32bf-4464-9804-28912fc7364e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:04.310464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1302/2595] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:38:04 2022] DELETE /identity/v3/projects/00b6edd7fbd1495a974ee7dce40c3edd => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:04.315198 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d56805-461b-46bf-900d-adafc238f2fe None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:04.315516 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d56805-461b-46bf-900d-adafc238f2fe None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:04.315839 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d56805-461b-46bf-900d-adafc238f2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:04.322556 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09097429-14be-4ef7-aa28-c4a65615b88d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:04.324205 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09097429-14be-4ef7-aa28-c4a65615b88d service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:04.333650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1294/2596] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:38:04.338829 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-20d56805-461b-46bf-900d-adafc238f2fe None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:04.340187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-20d56805-461b-46bf-900d-adafc238f2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:04.410941 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1303/2597] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:38:04.420356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7e9bf-ac8d-4a7e-9f62-b63a1859bc36 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:04.420647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7e9bf-ac8d-4a7e-9f62-b63a1859bc36 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:04.420925 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7e9bf-ac8d-4a7e-9f62-b63a1859bc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:04.442744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-20c7e9bf-ac8d-4a7e-9f62-b63a1859bc36 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:04.444008 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-20c7e9bf-ac8d-4a7e-9f62-b63a1859bc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:04.452369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1295/2598] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:38:04.457714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d56805-461b-46bf-900d-adafc238f2fe None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:04.467666 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d56805-461b-46bf-900d-adafc238f2fe None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:04.473751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015dc53-8c7d-464f-b073-8355d7ff7231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:04.475803 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015dc53-8c7d-464f-b073-8355d7ff7231 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:04.475803 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015dc53-8c7d-464f-b073-8355d7ff7231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:04.475803 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015dc53-8c7d-464f-b073-8355d7ff7231 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:04.475803 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015dc53-8c7d-464f-b073-8355d7ff7231 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:38:04.476324 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015dc53-8c7d-464f-b073-8355d7ff7231 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:04.477727 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015dc53-8c7d-464f-b073-8355d7ff7231 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:04.492627 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1304/2599] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:38:04.501875 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7e9bf-ac8d-4a7e-9f62-b63a1859bc36 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:04.511545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7e9bf-ac8d-4a7e-9f62-b63a1859bc36 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:04.516438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626edde-d84f-4ec0-aa22-c4aca5c7d510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:04.521729 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626edde-d84f-4ec0-aa22-c4aca5c7d510 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:04.521729 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626edde-d84f-4ec0-aa22-c4aca5c7d510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:04.521729 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626edde-d84f-4ec0-aa22-c4aca5c7d510 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:04.521729 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626edde-d84f-4ec0-aa22-c4aca5c7d510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:04.521729 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626edde-d84f-4ec0-aa22-c4aca5c7d510 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:04.639762 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1296/2600] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:38:04 2022] POST /identity/v3/projects => generated 365 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:04.644161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015dc53-8c7d-464f-b073-8355d7ff7231 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:04.655091 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015dc53-8c7d-464f-b073-8355d7ff7231 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:04.714055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be75bc-b4e4-4b73-b5bf-30607a4ba319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:04.714665 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be75bc-b4e4-4b73-b5bf-30607a4ba319 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:04.714984 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be75bc-b4e4-4b73-b5bf-30607a4ba319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:04.715267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be75bc-b4e4-4b73-b5bf-30607a4ba319 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:04.715907 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be75bc-b4e4-4b73-b5bf-30607a4ba319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:04.717837 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be75bc-b4e4-4b73-b5bf-30607a4ba319 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:04.825517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1305/2601] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:38:04 2022] POST /identity/v3/users => generated 283 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:04.827849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626edde-d84f-4ec0-aa22-c4aca5c7d510 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:04.840051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626edde-d84f-4ec0-aa22-c4aca5c7d510 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:04.844957 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81547c04-df96-48a3-9c22-4d1ee466ba18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:04.845400 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81547c04-df96-48a3-9c22-4d1ee466ba18 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:04.846597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81547c04-df96-48a3-9c22-4d1ee466ba18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:04.846597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81547c04-df96-48a3-9c22-4d1ee466ba18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:04.846597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81547c04-df96-48a3-9c22-4d1ee466ba18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:04.848571 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81547c04-df96-48a3-9c22-4d1ee466ba18 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:04.859148 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1297/2602] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:04.863544 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be75bc-b4e4-4b73-b5bf-30607a4ba319 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:04.874193 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be75bc-b4e4-4b73-b5bf-30607a4ba319 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:04.879502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b4299-4766-4b29-abcf-158b6205b89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:04.880327 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b4299-4766-4b29-abcf-158b6205b89e admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:04.880613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b4299-4766-4b29-abcf-158b6205b89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:04.880922 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b4299-4766-4b29-abcf-158b6205b89e admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:04.881575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b4299-4766-4b29-abcf-158b6205b89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:04.883425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b4299-4766-4b29-abcf-158b6205b89e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:04.895516 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1306/2603] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:04.902797 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81547c04-df96-48a3-9c22-4d1ee466ba18 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:04.913313 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81547c04-df96-48a3-9c22-4d1ee466ba18 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:04.918031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da54f12-9acd-4a08-b6cc-63c4e7d5fc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:04.918689 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da54f12-9acd-4a08-b6cc-63c4e7d5fc1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:04.919364 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da54f12-9acd-4a08-b6cc-63c4e7d5fc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:04.919501 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da54f12-9acd-4a08-b6cc-63c4e7d5fc1c admin admin] PATH_INFO: `/v3/projects/b8727644b60740de9ddf120d4d17122c/users/230884f06ffe4e3a8135e02844043358/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:04.934703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da54f12-9acd-4a08-b6cc-63c4e7d5fc1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8727644b60740de9ddf120d4d17122c, user_id=230884f06ffe4e3a8135e02844043358, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:04.937389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da54f12-9acd-4a08-b6cc-63c4e7d5fc1c admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:04.998741 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1298/2604] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:38:04 2022] PUT /identity/v3/projects/b8727644b60740de9ddf120d4d17122c/users/230884f06ffe4e3a8135e02844043358/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:05.009709 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea46186-36c3-4b58-a166-1cf35a2d4c6e None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:05.010085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea46186-36c3-4b58-a166-1cf35a2d4c6e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:05.010365 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea46186-36c3-4b58-a166-1cf35a2d4c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:05.039201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-5ea46186-36c3-4b58-a166-1cf35a2d4c6e None None] MFA Rules not processed for user `230884f06ffe4e3a8135e02844043358`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:05.040447 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-5ea46186-36c3-4b58-a166-1cf35a2d4c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:05.067519 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:05.105617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1307/2605] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:05.116297 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da54f12-9acd-4a08-b6cc-63c4e7d5fc1c admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:05.173550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:05.180071 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da54f12-9acd-4a08-b6cc-63c4e7d5fc1c admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:05.184800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeaacbb-3008-473f-a558-06b48219943c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:05.185406 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeaacbb-3008-473f-a558-06b48219943c admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:05.185690 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeaacbb-3008-473f-a558-06b48219943c admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:05.186028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeaacbb-3008-473f-a558-06b48219943c admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:05.186673 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeaacbb-3008-473f-a558-06b48219943c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:05.188748 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeaacbb-3008-473f-a558-06b48219943c admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:05.257692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1299/2606] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:38:05 2022] POST /identity/v3/projects => generated 369 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:05.261792 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea46186-36c3-4b58-a166-1cf35a2d4c6e None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:05.273191 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea46186-36c3-4b58-a166-1cf35a2d4c6e None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:05.332657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfb005-1f51-4b66-aea7-0b91bc6627e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:05.333275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfb005-1f51-4b66-aea7-0b91bc6627e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:05.333561 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfb005-1f51-4b66-aea7-0b91bc6627e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:05.333836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfb005-1f51-4b66-aea7-0b91bc6627e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:05.334512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfb005-1f51-4b66-aea7-0b91bc6627e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:05.336466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfb005-1f51-4b66-aea7-0b91bc6627e3 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:05.421491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1308/2607] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:38:05 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:05.426561 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeaacbb-3008-473f-a558-06b48219943c admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:05.437432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeaacbb-3008-473f-a558-06b48219943c admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:05.442248 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973287e9-7e0c-4560-aac4-7f3aaa1cc17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:05.442857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973287e9-7e0c-4560-aac4-7f3aaa1cc17a admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:05.443165 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973287e9-7e0c-4560-aac4-7f3aaa1cc17a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:05.443442 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973287e9-7e0c-4560-aac4-7f3aaa1cc17a admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:05.444092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973287e9-7e0c-4560-aac4-7f3aaa1cc17a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:05.445683 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973287e9-7e0c-4560-aac4-7f3aaa1cc17a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:05.456998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1300/2608] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:05.462620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfb005-1f51-4b66-aea7-0b91bc6627e3 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:05.473341 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfb005-1f51-4b66-aea7-0b91bc6627e3 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:05.478123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d55c2-c809-4982-baff-030122a9040c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:05.478781 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d55c2-c809-4982-baff-030122a9040c admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:05.479107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d55c2-c809-4982-baff-030122a9040c admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:05.479388 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d55c2-c809-4982-baff-030122a9040c admin admin] PATH_INFO: `/v3/projects/4a637ec178b84ea682e2960492faaca4/users/5943b63b4eee481c97a935b3db5503f0/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:05.494960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d55c2-c809-4982-baff-030122a9040c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a637ec178b84ea682e2960492faaca4, user_id=5943b63b4eee481c97a935b3db5503f0, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:05.497614 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d55c2-c809-4982-baff-030122a9040c admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:05.553348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1309/2609] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:38:05 2022] PUT /identity/v3/projects/4a637ec178b84ea682e2960492faaca4/users/5943b63b4eee481c97a935b3db5503f0/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:05.557580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973287e9-7e0c-4560-aac4-7f3aaa1cc17a admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:05.568209 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973287e9-7e0c-4560-aac4-7f3aaa1cc17a admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:05.630069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f702e5-5965-4e9e-9dbd-9aa708437618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:05.630676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f702e5-5965-4e9e-9dbd-9aa708437618 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:05.631050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f702e5-5965-4e9e-9dbd-9aa708437618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:05.631322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f702e5-5965-4e9e-9dbd-9aa708437618 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:05.631963 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f702e5-5965-4e9e-9dbd-9aa708437618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:05.633552 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f702e5-5965-4e9e-9dbd-9aa708437618 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:05.644480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1301/2610] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:38:05.648716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d55c2-c809-4982-baff-030122a9040c admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:05.659246 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d55c2-c809-4982-baff-030122a9040c admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:05.664829 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a9a5a-db12-4b1e-ad98-73361728adfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:05.665500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a9a5a-db12-4b1e-ad98-73361728adfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:05.665783 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a9a5a-db12-4b1e-ad98-73361728adfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:05.666058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a9a5a-db12-4b1e-ad98-73361728adfc admin admin] PATH_INFO: `/v3/projects/4a637ec178b84ea682e2960492faaca4/users/5943b63b4eee481c97a935b3db5503f0/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:05.668015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a9a5a-db12-4b1e-ad98-73361728adfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a637ec178b84ea682e2960492faaca4, user_id=5943b63b4eee481c97a935b3db5503f0, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:05.670668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a9a5a-db12-4b1e-ad98-73361728adfc admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:05.682862 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1310/2611] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:38:05 2022] PUT /identity/v3/projects/4a637ec178b84ea682e2960492faaca4/users/5943b63b4eee481c97a935b3db5503f0/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:38:05.694493 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb12418-7118-405d-af1e-010bb21c64f3 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:05.694791 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb12418-7118-405d-af1e-010bb21c64f3 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:05.700313 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb12418-7118-405d-af1e-010bb21c64f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:05.726603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-edb12418-7118-405d-af1e-010bb21c64f3 None None] MFA Rules not processed for user `5943b63b4eee481c97a935b3db5503f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:05.727914 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-edb12418-7118-405d-af1e-010bb21c64f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:05.795754 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1302/2612] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:05.807261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a9a5a-db12-4b1e-ad98-73361728adfc admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:05.809009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9c7a9a5a-db12-4b1e-ad98-73361728adfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:05.867770 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a9a5a-db12-4b1e-ad98-73361728adfc admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:05.871405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb65d91-8362-4a18-bddb-574602ab174b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:05.871996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb65d91-8362-4a18-bddb-574602ab174b service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:05.872279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb65d91-8362-4a18-bddb-574602ab174b service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:05.872554 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb65d91-8362-4a18-bddb-574602ab174b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:05.881801 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb65d91-8362-4a18-bddb-574602ab174b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:05.883272 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb65d91-8362-4a18-bddb-574602ab174b service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:05.892729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1311/2613] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:38:08.643017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb12418-7118-405d-af1e-010bb21c64f3 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:08.693676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:08.705151 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb12418-7118-405d-af1e-010bb21c64f3 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:08.710575 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f757a5-ee3c-4d48-b6d4-eae3a61406b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:08.711228 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f757a5-ee3c-4d48-b6d4-eae3a61406b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:08.711513 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f757a5-ee3c-4d48-b6d4-eae3a61406b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:08.711798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f757a5-ee3c-4d48-b6d4-eae3a61406b2 admin admin] PATH_INFO: `/v3/users/230884f06ffe4e3a8135e02844043358` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:08.712979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f757a5-ee3c-4d48-b6d4-eae3a61406b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=230884f06ffe4e3a8135e02844043358)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:08.715330 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f757a5-ee3c-4d48-b6d4-eae3a61406b2 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:08.875248 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-15f757a5-ee3c-4d48-b6d4-eae3a61406b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '230884f06ffe4e3a8135e02844043358'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:08.945977 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-15f757a5-ee3c-4d48-b6d4-eae3a61406b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '230884f06ffe4e3a8135e02844043358'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:08.946587 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-15f757a5-ee3c-4d48-b6d4-eae3a61406b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '230884f06ffe4e3a8135e02844043358'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:08.959575 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-15f757a5-ee3c-4d48-b6d4-eae3a61406b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '230884f06ffe4e3a8135e02844043358'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:08.961111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-15f757a5-ee3c-4d48-b6d4-eae3a61406b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '230884f06ffe4e3a8135e02844043358'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:08.973333 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1303/2614] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:38:08 2022] DELETE /identity/v3/users/230884f06ffe4e3a8135e02844043358 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:08.977887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9da91d-2a90-4fae-8e5f-9428c567c52b None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:08.978384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9da91d-2a90-4fae-8e5f-9428c567c52b None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:08.978505 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9da91d-2a90-4fae-8e5f-9428c567c52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:08.998800 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-bf9da91d-2a90-4fae-8e5f-9428c567c52b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:09.000031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-bf9da91d-2a90-4fae-8e5f-9428c567c52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:09.017187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:09.058640 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1312/2615] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:09.069652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f757a5-ee3c-4d48-b6d4-eae3a61406b2 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:09.071523 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-15f757a5-ee3c-4d48-b6d4-eae3a61406b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:09.129536 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f757a5-ee3c-4d48-b6d4-eae3a61406b2 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:09.133273 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8d4f1-baca-4693-b91a-f451e403f645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:09.133879 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8d4f1-baca-4693-b91a-f451e403f645 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:09.134160 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8d4f1-baca-4693-b91a-f451e403f645 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:09.134434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8d4f1-baca-4693-b91a-f451e403f645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:09.143266 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8d4f1-baca-4693-b91a-f451e403f645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:09.144832 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8d4f1-baca-4693-b91a-f451e403f645 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:09.154148 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1304/2616] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:38:09.513130 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9da91d-2a90-4fae-8e5f-9428c567c52b None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:09.514687 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-bf9da91d-2a90-4fae-8e5f-9428c567c52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:09.531774 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9da91d-2a90-4fae-8e5f-9428c567c52b None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:09.538731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cea33-f07d-43c8-a167-1ffb55e15951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:09.539583 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cea33-f07d-43c8-a167-1ffb55e15951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:09.539675 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cea33-f07d-43c8-a167-1ffb55e15951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:09.539956 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cea33-f07d-43c8-a167-1ffb55e15951 admin admin] PATH_INFO: `/v3/projects/b8727644b60740de9ddf120d4d17122c` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:09.541239 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57cea33-f07d-43c8-a167-1ffb55e15951 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8727644b60740de9ddf120d4d17122c)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:09.543219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57cea33-f07d-43c8-a167-1ffb55e15951 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:09.645297 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a57cea33-f07d-43c8-a167-1ffb55e15951 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8727644b60740de9ddf120d4d17122c', 'request_id': 'req-a57cea33-f07d-43c8-a167-1ffb55e15951'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:09.651271 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a57cea33-f07d-43c8-a167-1ffb55e15951 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8727644b60740de9ddf120d4d17122c', 'request_id': 'req-a57cea33-f07d-43c8-a167-1ffb55e15951'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:09.695048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a57cea33-f07d-43c8-a167-1ffb55e15951 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8727644b60740de9ddf120d4d17122c', 'request_id': 'req-a57cea33-f07d-43c8-a167-1ffb55e15951'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:09.707546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a57cea33-f07d-43c8-a167-1ffb55e15951 admin admin] The token cache is being invalidate because project b8727644b60740de9ddf120d4d17122c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:38:09.707840 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a57cea33-f07d-43c8-a167-1ffb55e15951 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:09.708491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a57cea33-f07d-43c8-a167-1ffb55e15951 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:09.709985 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1313/2617] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:38:09 2022] DELETE /identity/v3/projects/b8727644b60740de9ddf120d4d17122c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:09.714923 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8d4f1-baca-4693-b91a-f451e403f645 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:09.724836 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8d4f1-baca-4693-b91a-f451e403f645 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:09.781212 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247e2f9-184d-47cf-83b6-db4d264598ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:09.781839 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247e2f9-184d-47cf-83b6-db4d264598ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:09.782120 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247e2f9-184d-47cf-83b6-db4d264598ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:09.782395 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247e2f9-184d-47cf-83b6-db4d264598ed admin admin] PATH_INFO: `/v3/users/5943b63b4eee481c97a935b3db5503f0` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:09.783517 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247e2f9-184d-47cf-83b6-db4d264598ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=5943b63b4eee481c97a935b3db5503f0)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:09.785621 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247e2f9-184d-47cf-83b6-db4d264598ed admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:09.799072 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:09.974703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c247e2f9-184d-47cf-83b6-db4d264598ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5943b63b4eee481c97a935b3db5503f0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:09.995165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ff0c2-e4ee-46d8-8e42-77aa514bbcb3 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:09.996202 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ff0c2-e4ee-46d8-8e42-77aa514bbcb3 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:09.996318 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ff0c2-e4ee-46d8-8e42-77aa514bbcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:10.015757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-c58ff0c2-e4ee-46d8-8e42-77aa514bbcb3 None None] MFA Rules not processed for user `1ee990b1658e49cc8bc2fe15e7dc36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:10.017006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c58ff0c2-e4ee-46d8-8e42-77aa514bbcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:10.066281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c247e2f9-184d-47cf-83b6-db4d264598ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5943b63b4eee481c97a935b3db5503f0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:10.066992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c247e2f9-184d-47cf-83b6-db4d264598ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5943b63b4eee481c97a935b3db5503f0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:10.082162 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c247e2f9-184d-47cf-83b6-db4d264598ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5943b63b4eee481c97a935b3db5503f0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:10.083009 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c247e2f9-184d-47cf-83b6-db4d264598ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5943b63b4eee481c97a935b3db5503f0'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:10.088107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1314/2618] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 31 19:38:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:38:10.107731 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1305/2619] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:38:09 2022] DELETE /identity/v3/users/5943b63b4eee481c97a935b3db5503f0 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:10.260778 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ff0c2-e4ee-46d8-8e42-77aa514bbcb3 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:10.260914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c58ff0c2-e4ee-46d8-8e42-77aa514bbcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:10.351466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:10.383194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ff0c2-e4ee-46d8-8e42-77aa514bbcb3 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:10.444707 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85158628-fc5d-4a6a-a547-cfe4e8efaa51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25ed58369ffd46e8b7fa1880662926e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:10.445324 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85158628-fc5d-4a6a-a547-cfe4e8efaa51 service swift] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:10.445604 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85158628-fc5d-4a6a-a547-cfe4e8efaa51 service swift] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:10.445876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85158628-fc5d-4a6a-a547-cfe4e8efaa51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:10.447657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-85158628-fc5d-4a6a-a547-cfe4e8efaa51 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:10.507814 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85158628-fc5d-4a6a-a547-cfe4e8efaa51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:10.509482 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85158628-fc5d-4a6a-a547-cfe4e8efaa51 service swift] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:10.524592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1315/2620] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 31 19:38:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:38:10.540037 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247e2f9-184d-47cf-83b6-db4d264598ed admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:10.542543 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c247e2f9-184d-47cf-83b6-db4d264598ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:10.650157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247e2f9-184d-47cf-83b6-db4d264598ed admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:10.653906 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3e101-a675-47ec-90b9-62f077275425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:10.654673 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3e101-a675-47ec-90b9-62f077275425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:10.655055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3e101-a675-47ec-90b9-62f077275425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:10.655407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3e101-a675-47ec-90b9-62f077275425 admin admin] PATH_INFO: `/v3/projects/4a637ec178b84ea682e2960492faaca4` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:10.656718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3e101-a675-47ec-90b9-62f077275425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a637ec178b84ea682e2960492faaca4)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:10.658855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3e101-a675-47ec-90b9-62f077275425 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:10.769070 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6cf3e101-a675-47ec-90b9-62f077275425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a637ec178b84ea682e2960492faaca4', 'request_id': 'req-6cf3e101-a675-47ec-90b9-62f077275425'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:10.779532 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6cf3e101-a675-47ec-90b9-62f077275425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a637ec178b84ea682e2960492faaca4', 'request_id': 'req-6cf3e101-a675-47ec-90b9-62f077275425'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:10.793763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6cf3e101-a675-47ec-90b9-62f077275425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a637ec178b84ea682e2960492faaca4', 'request_id': 'req-6cf3e101-a675-47ec-90b9-62f077275425'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:10.878294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6cf3e101-a675-47ec-90b9-62f077275425 admin admin] The token cache is being invalidate because project 4a637ec178b84ea682e2960492faaca4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:38:10.878632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6cf3e101-a675-47ec-90b9-62f077275425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:10.880026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6cf3e101-a675-47ec-90b9-62f077275425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:10.882339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1306/2621] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:38:10 2022] DELETE /identity/v3/projects/4a637ec178b84ea682e2960492faaca4 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:10.887053 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047715ec-d0b3-4740-bb7d-f8a3ef79b1a6 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:10.887547 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047715ec-d0b3-4740-bb7d-f8a3ef79b1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:10.887991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047715ec-d0b3-4740-bb7d-f8a3ef79b1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:10.918791 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-047715ec-d0b3-4740-bb7d-f8a3ef79b1a6 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:10.920778 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-047715ec-d0b3-4740-bb7d-f8a3ef79b1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:11.050770 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1316/2622] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:11.060165 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34613b9b-7de8-449f-a663-7fe3b25045ca None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:11.060466 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34613b9b-7de8-449f-a663-7fe3b25045ca None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:11.060705 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34613b9b-7de8-449f-a663-7fe3b25045ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:11.117925 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-34613b9b-7de8-449f-a663-7fe3b25045ca None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:11.123592 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-34613b9b-7de8-449f-a663-7fe3b25045ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:11.143086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1307/2623] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:11.147346 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047715ec-d0b3-4740-bb7d-f8a3ef79b1a6 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:11.162991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047715ec-d0b3-4740-bb7d-f8a3ef79b1a6 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:11.172317 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79dde5-01ea-44c3-90a5-1a6d29e2f2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:11.173534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79dde5-01ea-44c3-90a5-1a6d29e2f2e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:11.174383 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79dde5-01ea-44c3-90a5-1a6d29e2f2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:11.174946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79dde5-01ea-44c3-90a5-1a6d29e2f2e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:11.175858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79dde5-01ea-44c3-90a5-1a6d29e2f2e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:38:11.176175 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79dde5-01ea-44c3-90a5-1a6d29e2f2e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:11.179135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79dde5-01ea-44c3-90a5-1a6d29e2f2e3 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:11.196170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1317/2624] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:38:11.201140 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34613b9b-7de8-449f-a663-7fe3b25045ca None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:11.212827 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34613b9b-7de8-449f-a663-7fe3b25045ca None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:11.218439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30861320-88a3-486e-9cd7-658bba212ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:11.219229 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30861320-88a3-486e-9cd7-658bba212ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:11.219653 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30861320-88a3-486e-9cd7-658bba212ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:11.219995 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30861320-88a3-486e-9cd7-658bba212ab6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:11.220824 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30861320-88a3-486e-9cd7-658bba212ab6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:11.230613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30861320-88a3-486e-9cd7-658bba212ab6 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:11.253999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:11.341477 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1308/2625] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:38:11 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:11.352796 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79dde5-01ea-44c3-90a5-1a6d29e2f2e3 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:11.385881 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79dde5-01ea-44c3-90a5-1a6d29e2f2e3 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:11.404020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:11.498855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a1763-d682-43ac-b008-7abbeff2f0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:11.499636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a1763-d682-43ac-b008-7abbeff2f0be admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:11.499974 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a1763-d682-43ac-b008-7abbeff2f0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:11.500246 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a1763-d682-43ac-b008-7abbeff2f0be admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:11.500925 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a1763-d682-43ac-b008-7abbeff2f0be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:11.507928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a1763-d682-43ac-b008-7abbeff2f0be admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:11.606494 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1318/2626] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:38:11 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:11.609179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30861320-88a3-486e-9cd7-658bba212ab6 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:11.625004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30861320-88a3-486e-9cd7-658bba212ab6 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:11.632127 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4048d-3c55-4ab1-b827-9829b85fbdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:11.632957 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4048d-3c55-4ab1-b827-9829b85fbdfb admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:11.633407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4048d-3c55-4ab1-b827-9829b85fbdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:11.633711 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4048d-3c55-4ab1-b827-9829b85fbdfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:11.634358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce4048d-3c55-4ab1-b827-9829b85fbdfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:11.636156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce4048d-3c55-4ab1-b827-9829b85fbdfb admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:11.660473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a1763-d682-43ac-b008-7abbeff2f0be admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:11.673740 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1309/2627] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:11.682080 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a1763-d682-43ac-b008-7abbeff2f0be admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:11.688512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f0426-566f-492a-9c2b-b5ac2dcad520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:11.690468 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f0426-566f-492a-9c2b-b5ac2dcad520 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:11.690958 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f0426-566f-492a-9c2b-b5ac2dcad520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:11.691521 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f0426-566f-492a-9c2b-b5ac2dcad520 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:11.713527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f0426-566f-492a-9c2b-b5ac2dcad520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:11.715627 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f0426-566f-492a-9c2b-b5ac2dcad520 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:11.752406 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4048d-3c55-4ab1-b827-9829b85fbdfb admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:11.758151 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1319/2628] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:11.781813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4048d-3c55-4ab1-b827-9829b85fbdfb admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:11.792601 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f617e6-3044-4dc2-a4e4-42de88673694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:11.795456 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f617e6-3044-4dc2-a4e4-42de88673694 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:11.795913 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f617e6-3044-4dc2-a4e4-42de88673694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:11.796203 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f617e6-3044-4dc2-a4e4-42de88673694 admin admin] PATH_INFO: `/v3/projects/b895afb5bb474114ac4fd846cc25929b/users/2ad203318a424276bb82773cb95b4d5f/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:11.840224 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f617e6-3044-4dc2-a4e4-42de88673694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b895afb5bb474114ac4fd846cc25929b, user_id=2ad203318a424276bb82773cb95b4d5f, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:11.843779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f617e6-3044-4dc2-a4e4-42de88673694 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:11.946936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1310/2629] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:38:11 2022] PUT /identity/v3/projects/b895afb5bb474114ac4fd846cc25929b/users/2ad203318a424276bb82773cb95b4d5f/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:11.947410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713d9e4-02a7-443d-a0d4-5ee09c89e087 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:11.947798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713d9e4-02a7-443d-a0d4-5ee09c89e087 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:11.948177 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713d9e4-02a7-443d-a0d4-5ee09c89e087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:11.994401 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-c713d9e4-02a7-443d-a0d4-5ee09c89e087 None None] MFA Rules not processed for user `2ad203318a424276bb82773cb95b4d5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:11.999503 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c713d9e4-02a7-443d-a0d4-5ee09c89e087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:12.003488 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f617e6-3044-4dc2-a4e4-42de88673694 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:12.005560 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e7f617e6-3044-4dc2-a4e4-42de88673694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:12.095471 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1320/2630] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:38:12.096918 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713d9e4-02a7-443d-a0d4-5ee09c89e087 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:12.191578 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f617e6-3044-4dc2-a4e4-42de88673694 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:12.206084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d886f-57d4-4b46-8fce-10019de9fcda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:12.208604 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d886f-57d4-4b46-8fce-10019de9fcda service placement] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:12.210638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d886f-57d4-4b46-8fce-10019de9fcda service placement] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:12.210940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d886f-57d4-4b46-8fce-10019de9fcda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:12.214673 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-3c1d886f-57d4-4b46-8fce-10019de9fcda service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:12.223654 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713d9e4-02a7-443d-a0d4-5ee09c89e087 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:12.235879 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d886f-57d4-4b46-8fce-10019de9fcda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:12.240527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a108c-9888-4a3c-b70d-8529f57a6bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:12.240527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a108c-9888-4a3c-b70d-8529f57a6bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:12.240527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a108c-9888-4a3c-b70d-8529f57a6bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:12.240527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a108c-9888-4a3c-b70d-8529f57a6bc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:12.240942 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a108c-9888-4a3c-b70d-8529f57a6bc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:12.242079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d886f-57d4-4b46-8fce-10019de9fcda service placement] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:12.242751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a108c-9888-4a3c-b70d-8529f57a6bc2 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:12.267327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1311/2631] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:38:12.383059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1321/2632] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:38:12 2022] POST /identity/v3/projects => generated 387 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:12.386751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d886f-57d4-4b46-8fce-10019de9fcda service placement] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:12.401421 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d886f-57d4-4b46-8fce-10019de9fcda service placement] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:12.508755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7820b0-0fec-44f9-8b59-ce03222dd78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:12.509648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7820b0-0fec-44f9-8b59-ce03222dd78b admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:12.510942 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7820b0-0fec-44f9-8b59-ce03222dd78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:12.511433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7820b0-0fec-44f9-8b59-ce03222dd78b admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:12.512229 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7820b0-0fec-44f9-8b59-ce03222dd78b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:12.518396 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7820b0-0fec-44f9-8b59-ce03222dd78b admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:12.612424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1312/2633] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:38:12 2022] POST /identity/v3/users => generated 300 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:38:12.613279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a108c-9888-4a3c-b70d-8529f57a6bc2 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:12.622044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:12.627621 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a108c-9888-4a3c-b70d-8529f57a6bc2 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:12.639038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df385877-6552-4723-8869-61e43d871ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:12.639976 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df385877-6552-4723-8869-61e43d871ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:12.640381 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df385877-6552-4723-8869-61e43d871ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:12.640762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df385877-6552-4723-8869-61e43d871ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:12.642504 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df385877-6552-4723-8869-61e43d871ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:12.644444 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df385877-6552-4723-8869-61e43d871ea4 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:12.674252 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1322/2634] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:12.676326 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7820b0-0fec-44f9-8b59-ce03222dd78b admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:12.689841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7820b0-0fec-44f9-8b59-ce03222dd78b admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:12.696304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4dff0-ba0f-4529-8ac8-f7cdefaf9e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:12.697147 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4dff0-ba0f-4529-8ac8-f7cdefaf9e1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:12.697531 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4dff0-ba0f-4529-8ac8-f7cdefaf9e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:12.697899 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4dff0-ba0f-4529-8ac8-f7cdefaf9e1e admin admin] PATH_INFO: `/v3/projects/d78b53b1754e4ddfaf39300a868f5f7d/users/1d106130ef344179a4c228086b2b7480/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:12.727403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c4dff0-ba0f-4529-8ac8-f7cdefaf9e1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d78b53b1754e4ddfaf39300a868f5f7d, user_id=1d106130ef344179a4c228086b2b7480, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:12.730523 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c4dff0-ba0f-4529-8ac8-f7cdefaf9e1e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:12.795444 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1313/2635] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:38:12 2022] PUT /identity/v3/projects/d78b53b1754e4ddfaf39300a868f5f7d/users/1d106130ef344179a4c228086b2b7480/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:12.799300 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df385877-6552-4723-8869-61e43d871ea4 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:12.812376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df385877-6552-4723-8869-61e43d871ea4 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:12.950860 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea21a7c-b64a-4960-98e4-8a91403cf4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:12.951659 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea21a7c-b64a-4960-98e4-8a91403cf4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:12.952165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea21a7c-b64a-4960-98e4-8a91403cf4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:12.952323 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea21a7c-b64a-4960-98e4-8a91403cf4a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:12.953218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea21a7c-b64a-4960-98e4-8a91403cf4a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:12.955121 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea21a7c-b64a-4960-98e4-8a91403cf4a2 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:12.980592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1323/2636] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:12.981584 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4dff0-ba0f-4529-8ac8-f7cdefaf9e1e admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:12.994227 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4dff0-ba0f-4529-8ac8-f7cdefaf9e1e admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:13.002012 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011cb7c1-6700-4c4e-977f-4c4d4e9039e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:13.003019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cb7c1-6700-4c4e-977f-4c4d4e9039e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:13.003487 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cb7c1-6700-4c4e-977f-4c4d4e9039e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:13.003930 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cb7c1-6700-4c4e-977f-4c4d4e9039e0 admin admin] PATH_INFO: `/v3/projects/d78b53b1754e4ddfaf39300a868f5f7d/users/1d106130ef344179a4c228086b2b7480/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:13.006494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011cb7c1-6700-4c4e-977f-4c4d4e9039e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d78b53b1754e4ddfaf39300a868f5f7d, user_id=1d106130ef344179a4c228086b2b7480, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:13.009179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011cb7c1-6700-4c4e-977f-4c4d4e9039e0 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:13.019340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:13.026572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1314/2637] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:38:12 2022] PUT /identity/v3/projects/d78b53b1754e4ddfaf39300a868f5f7d/users/1d106130ef344179a4c228086b2b7480/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:13.039859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4893e-bb02-4499-bed7-1f50aae571e3 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:13.040270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4893e-bb02-4499-bed7-1f50aae571e3 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:13.040617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4893e-bb02-4499-bed7-1f50aae571e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:13.074572 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-fac4893e-bb02-4499-bed7-1f50aae571e3 None None] MFA Rules not processed for user `1d106130ef344179a4c228086b2b7480`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:13.076767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-fac4893e-bb02-4499-bed7-1f50aae571e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:13.179375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1324/2638] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:13.187832 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011cb7c1-6700-4c4e-977f-4c4d4e9039e0 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:13.189037 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-011cb7c1-6700-4c4e-977f-4c4d4e9039e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:13.259334 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011cb7c1-6700-4c4e-977f-4c4d4e9039e0 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:13.263063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30081a-7458-4bc3-9938-7c44cbddbdcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:13.263676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30081a-7458-4bc3-9938-7c44cbddbdcd service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:13.263957 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30081a-7458-4bc3-9938-7c44cbddbdcd service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:13.264228 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30081a-7458-4bc3-9938-7c44cbddbdcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:13.275030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a30081a-7458-4bc3-9938-7c44cbddbdcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:13.276512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a30081a-7458-4bc3-9938-7c44cbddbdcd service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:13.288868 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1315/2639] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:38:13.373978 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac4893e-bb02-4499-bed7-1f50aae571e3 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:13.459610 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac4893e-bb02-4499-bed7-1f50aae571e3 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:13.464596 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925477a-d447-4a9f-b5c3-6a323ab15942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:13.465346 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925477a-d447-4a9f-b5c3-6a323ab15942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:13.465620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925477a-d447-4a9f-b5c3-6a323ab15942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:13.465863 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925477a-d447-4a9f-b5c3-6a323ab15942 admin admin] PATH_INFO: `/v3/users/2ad203318a424276bb82773cb95b4d5f` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:13.467165 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925477a-d447-4a9f-b5c3-6a323ab15942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ad203318a424276bb82773cb95b4d5f)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:13.469346 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925477a-d447-4a9f-b5c3-6a323ab15942 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:13.491883 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:13.712017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a925477a-d447-4a9f-b5c3-6a323ab15942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ad203318a424276bb82773cb95b4d5f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:13.736496 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a925477a-d447-4a9f-b5c3-6a323ab15942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ad203318a424276bb82773cb95b4d5f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:13.816375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a925477a-d447-4a9f-b5c3-6a323ab15942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ad203318a424276bb82773cb95b4d5f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:13.826581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a925477a-d447-4a9f-b5c3-6a323ab15942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ad203318a424276bb82773cb95b4d5f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:13.827381 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a925477a-d447-4a9f-b5c3-6a323ab15942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ad203318a424276bb82773cb95b4d5f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:13.836561 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1325/2640] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:38:13 2022] DELETE /identity/v3/users/2ad203318a424276bb82773cb95b4d5f => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:13.842494 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73db41d-4011-4be6-83d0-45363bbc9368 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:13.843272 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73db41d-4011-4be6-83d0-45363bbc9368 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:13.843272 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73db41d-4011-4be6-83d0-45363bbc9368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:13.864159 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-a73db41d-4011-4be6-83d0-45363bbc9368 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:13.865904 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a73db41d-4011-4be6-83d0-45363bbc9368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:13.939236 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1316/2641] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:38:13.950638 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925477a-d447-4a9f-b5c3-6a323ab15942 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:13.952282 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a925477a-d447-4a9f-b5c3-6a323ab15942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:14.021411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925477a-d447-4a9f-b5c3-6a323ab15942 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:14.025510 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a435d-f7f4-4dd0-bd59-a1735f6bfdbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:14.026130 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a435d-f7f4-4dd0-bd59-a1735f6bfdbf service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:14.026408 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a435d-f7f4-4dd0-bd59-a1735f6bfdbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:14.026678 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a435d-f7f4-4dd0-bd59-a1735f6bfdbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:14.037234 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a435d-f7f4-4dd0-bd59-a1735f6bfdbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:14.040779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a435d-f7f4-4dd0-bd59-a1735f6bfdbf service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:14.048885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1326/2642] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:38:14.651863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73db41d-4011-4be6-83d0-45363bbc9368 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:14.653325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a73db41d-4011-4be6-83d0-45363bbc9368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:14.672929 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73db41d-4011-4be6-83d0-45363bbc9368 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:14.684970 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:14.685625 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:14.685912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:14.686206 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5 admin admin] PATH_INFO: `/v3/projects/b895afb5bb474114ac4fd846cc25929b` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:14.689551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b895afb5bb474114ac4fd846cc25929b)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:14.690664 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:14.831686 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b895afb5bb474114ac4fd846cc25929b', 'request_id': 'req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:14.838327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b895afb5bb474114ac4fd846cc25929b', 'request_id': 'req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:14.850600 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b895afb5bb474114ac4fd846cc25929b', 'request_id': 'req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:14.977898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5 admin admin] The token cache is being invalidate because project b895afb5bb474114ac4fd846cc25929b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:38:14.978340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:14.979142 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:14.981540 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1317/2643] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:38:14 2022] DELETE /identity/v3/projects/b895afb5bb474114ac4fd846cc25929b => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:14.985581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a435d-f7f4-4dd0-bd59-a1735f6bfdbf service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:14.999256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a435d-f7f4-4dd0-bd59-a1735f6bfdbf service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:15.083146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4313246-cb77-425e-957f-016fa3b342cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:15.083146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4313246-cb77-425e-957f-016fa3b342cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:15.083146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4313246-cb77-425e-957f-016fa3b342cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:15.083146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4313246-cb77-425e-957f-016fa3b342cb admin admin] PATH_INFO: `/v3/users/1d106130ef344179a4c228086b2b7480` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:15.084067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4313246-cb77-425e-957f-016fa3b342cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d106130ef344179a4c228086b2b7480)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:15.084067 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4313246-cb77-425e-957f-016fa3b342cb admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:15.266273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:15.275794 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e4313246-cb77-425e-957f-016fa3b342cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d106130ef344179a4c228086b2b7480'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:15.291891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e4313246-cb77-425e-957f-016fa3b342cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d106130ef344179a4c228086b2b7480'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:15.346927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e4313246-cb77-425e-957f-016fa3b342cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d106130ef344179a4c228086b2b7480'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:15.357115 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e4313246-cb77-425e-957f-016fa3b342cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d106130ef344179a4c228086b2b7480'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:15.357915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e4313246-cb77-425e-957f-016fa3b342cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d106130ef344179a4c228086b2b7480'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:15.364882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1327/2644] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:38:14 2022] DELETE /identity/v3/users/1d106130ef344179a4c228086b2b7480 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:15.887510 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:15.890317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:15.919334 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87f4bf-a254-4043-b4ba-e56b58bd08e5 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:15.948365 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:15.991519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b70bdd-18e4-4584-9803-14722b20b8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:15.992275 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b70bdd-18e4-4584-9803-14722b20b8af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:15.992560 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b70bdd-18e4-4584-9803-14722b20b8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:15.992837 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b70bdd-18e4-4584-9803-14722b20b8af admin admin] PATH_INFO: `/v3/projects/d78b53b1754e4ddfaf39300a868f5f7d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:15.993960 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b70bdd-18e4-4584-9803-14722b20b8af admin admin] RBAC: Authorizing `identity:delete_project(project_id=d78b53b1754e4ddfaf39300a868f5f7d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:15.997963 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b70bdd-18e4-4584-9803-14722b20b8af admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:16.121698 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-20b70bdd-18e4-4584-9803-14722b20b8af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd78b53b1754e4ddfaf39300a868f5f7d', 'request_id': 'req-20b70bdd-18e4-4584-9803-14722b20b8af'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:16.131550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-20b70bdd-18e4-4584-9803-14722b20b8af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd78b53b1754e4ddfaf39300a868f5f7d', 'request_id': 'req-20b70bdd-18e4-4584-9803-14722b20b8af'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:16.151189 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-20b70bdd-18e4-4584-9803-14722b20b8af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd78b53b1754e4ddfaf39300a868f5f7d', 'request_id': 'req-20b70bdd-18e4-4584-9803-14722b20b8af'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:16.245869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-20b70bdd-18e4-4584-9803-14722b20b8af admin admin] The token cache is being invalidate because project d78b53b1754e4ddfaf39300a868f5f7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:38:16.246198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-20b70bdd-18e4-4584-9803-14722b20b8af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:16.246853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-20b70bdd-18e4-4584-9803-14722b20b8af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:16.248829 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1318/2645] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:38:15 2022] DELETE /identity/v3/projects/d78b53b1754e4ddfaf39300a868f5f7d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:16.253767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f8e62-930d-4bf5-a9a4-820f9124ef3e None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:16.255332 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f8e62-930d-4bf5-a9a4-820f9124ef3e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:16.255756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f8e62-930d-4bf5-a9a4-820f9124ef3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:16.280425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-651f8e62-930d-4bf5-a9a4-820f9124ef3e None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:16.281991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-651f8e62-930d-4bf5-a9a4-820f9124ef3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:16.356260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1328/2646] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:16.365239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3719a-3aef-48c4-bdff-bdac5c3bea23 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:16.365535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3719a-3aef-48c4-bdff-bdac5c3bea23 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:16.365808 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3719a-3aef-48c4-bdff-bdac5c3bea23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:16.396376 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-a3e3719a-3aef-48c4-bdff-bdac5c3bea23 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:16.398016 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a3e3719a-3aef-48c4-bdff-bdac5c3bea23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:16.412903 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f8e62-930d-4bf5-a9a4-820f9124ef3e None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:16.419050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1319/2647] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:16.424514 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f8e62-930d-4bf5-a9a4-820f9124ef3e None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:16.430257 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde9acc-6ac7-4263-84b7-309a591ce1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:16.431027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde9acc-6ac7-4263-84b7-309a591ce1cd admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:16.431412 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde9acc-6ac7-4263-84b7-309a591ce1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:16.432130 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde9acc-6ac7-4263-84b7-309a591ce1cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:16.432819 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde9acc-6ac7-4263-84b7-309a591ce1cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:38:16.433977 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde9acc-6ac7-4263-84b7-309a591ce1cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:16.435085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde9acc-6ac7-4263-84b7-309a591ce1cd admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:16.452258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1329/2648] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:38:16.453195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3719a-3aef-48c4-bdff-bdac5c3bea23 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:16.463840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3719a-3aef-48c4-bdff-bdac5c3bea23 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:16.468699 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478eace9-8449-47e8-a59f-dc001b6e49c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:16.469308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eace9-8449-47e8-a59f-dc001b6e49c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:16.469590 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eace9-8449-47e8-a59f-dc001b6e49c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:16.469862 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eace9-8449-47e8-a59f-dc001b6e49c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:16.470499 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478eace9-8449-47e8-a59f-dc001b6e49c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:16.472437 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478eace9-8449-47e8-a59f-dc001b6e49c4 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:16.544537 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1320/2649] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:38:16 2022] POST /identity/v3/projects => generated 367 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:38:16.549581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde9acc-6ac7-4263-84b7-309a591ce1cd admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:16.562803 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde9acc-6ac7-4263-84b7-309a591ce1cd admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:16.628441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f677f1-07de-4b34-8534-a589b7eafab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:16.630031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f677f1-07de-4b34-8534-a589b7eafab8 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:16.630660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f677f1-07de-4b34-8534-a589b7eafab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:16.631175 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f677f1-07de-4b34-8534-a589b7eafab8 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:16.631986 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f677f1-07de-4b34-8534-a589b7eafab8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:16.637517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f677f1-07de-4b34-8534-a589b7eafab8 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:16.721694 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1330/2650] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:38:16 2022] POST /identity/v3/users => generated 284 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:16.723860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478eace9-8449-47e8-a59f-dc001b6e49c4 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:16.737310 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478eace9-8449-47e8-a59f-dc001b6e49c4 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:16.743718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64513771-3696-4e8a-a1f6-1d5fc935643e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:16.745053 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64513771-3696-4e8a-a1f6-1d5fc935643e admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:16.745341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64513771-3696-4e8a-a1f6-1d5fc935643e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:16.746366 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64513771-3696-4e8a-a1f6-1d5fc935643e admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:16.747242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64513771-3696-4e8a-a1f6-1d5fc935643e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:16.749632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64513771-3696-4e8a-a1f6-1d5fc935643e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:16.766429 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1321/2651] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:16.769460 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f677f1-07de-4b34-8534-a589b7eafab8 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:16.785170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f677f1-07de-4b34-8534-a589b7eafab8 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:16.799494 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c5b32-7d90-491e-9e00-c07b5662c12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:16.800283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c5b32-7d90-491e-9e00-c07b5662c12a admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:16.800643 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c5b32-7d90-491e-9e00-c07b5662c12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:16.801010 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c5b32-7d90-491e-9e00-c07b5662c12a admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:16.801901 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c5b32-7d90-491e-9e00-c07b5662c12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:16.804355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c5b32-7d90-491e-9e00-c07b5662c12a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:16.823901 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1331/2652] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:16.829466 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64513771-3696-4e8a-a1f6-1d5fc935643e admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:16.842623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64513771-3696-4e8a-a1f6-1d5fc935643e admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:16.848059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3581b-17a1-416e-b82b-595574c79b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:16.848884 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3581b-17a1-416e-b82b-595574c79b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:16.849151 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3581b-17a1-416e-b82b-595574c79b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:16.850271 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3581b-17a1-416e-b82b-595574c79b8e admin admin] PATH_INFO: `/v3/projects/9579dcaed8eb4e79988b7e5e11e272b6/users/3a71a5ff3e404920bda6e5affc0add63/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:16.866949 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3581b-17a1-416e-b82b-595574c79b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9579dcaed8eb4e79988b7e5e11e272b6, user_id=3a71a5ff3e404920bda6e5affc0add63, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:16.870655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3581b-17a1-416e-b82b-595574c79b8e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:16.991906 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1322/2653] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:38:16 2022] PUT /identity/v3/projects/9579dcaed8eb4e79988b7e5e11e272b6/users/3a71a5ff3e404920bda6e5affc0add63/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:16.994424 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b579bf-8664-4e56-9a87-09f269e033e7 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:16.995313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b579bf-8664-4e56-9a87-09f269e033e7 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:16.995751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b579bf-8664-4e56-9a87-09f269e033e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:17.028821 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-e1b579bf-8664-4e56-9a87-09f269e033e7 None None] MFA Rules not processed for user `3a71a5ff3e404920bda6e5affc0add63`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:17.030403 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e1b579bf-8664-4e56-9a87-09f269e033e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:17.041347 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:17.107945 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1332/2654] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:38:17.112518 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3581b-17a1-416e-b82b-595574c79b8e admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:17.183036 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:17.195271 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3581b-17a1-416e-b82b-595574c79b8e admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:17.200401 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4000b4-797a-4d74-97fa-8fc7176f0643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:17.201049 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4000b4-797a-4d74-97fa-8fc7176f0643 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:17.201354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4000b4-797a-4d74-97fa-8fc7176f0643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:17.201652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4000b4-797a-4d74-97fa-8fc7176f0643 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:17.202344 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4000b4-797a-4d74-97fa-8fc7176f0643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:17.204485 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4000b4-797a-4d74-97fa-8fc7176f0643 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:17.605900 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1323/2655] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:38:17 2022] POST /identity/v3/projects => generated 365 bytes in 494 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:17.610805 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b579bf-8664-4e56-9a87-09f269e033e7 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:17.623506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b579bf-8664-4e56-9a87-09f269e033e7 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:17.688992 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884235b-76ab-4df3-ac85-437439fe1a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:17.689682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884235b-76ab-4df3-ac85-437439fe1a3f admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:17.690031 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884235b-76ab-4df3-ac85-437439fe1a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:17.690782 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884235b-76ab-4df3-ac85-437439fe1a3f admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:17.691186 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884235b-76ab-4df3-ac85-437439fe1a3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:17.693293 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884235b-76ab-4df3-ac85-437439fe1a3f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:17.784921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1333/2656] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:38:17 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:38:17.790215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4000b4-797a-4d74-97fa-8fc7176f0643 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:17.801937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4000b4-797a-4d74-97fa-8fc7176f0643 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:17.808322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8ce26-bb20-4235-ae13-6286adb74902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:17.808965 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8ce26-bb20-4235-ae13-6286adb74902 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:17.809370 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8ce26-bb20-4235-ae13-6286adb74902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:17.809666 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8ce26-bb20-4235-ae13-6286adb74902 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:17.810363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b8ce26-bb20-4235-ae13-6286adb74902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:17.812037 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b8ce26-bb20-4235-ae13-6286adb74902 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:17.841249 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1324/2657] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:17.843145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884235b-76ab-4df3-ac85-437439fe1a3f admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:17.856809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884235b-76ab-4df3-ac85-437439fe1a3f admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:17.862545 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddf476-105d-42f3-a19d-bf260f823498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:17.863260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddf476-105d-42f3-a19d-bf260f823498 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:17.863711 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddf476-105d-42f3-a19d-bf260f823498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:17.863908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddf476-105d-42f3-a19d-bf260f823498 admin admin] PATH_INFO: `/v3/projects/2b095e38cd4b4f3c9e8c3ab2dbd4bb90/users/f0115f43efba4e5ba90b4b8a9d8e4a21/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:17.882482 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ddf476-105d-42f3-a19d-bf260f823498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b095e38cd4b4f3c9e8c3ab2dbd4bb90, user_id=f0115f43efba4e5ba90b4b8a9d8e4a21, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:17.885196 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ddf476-105d-42f3-a19d-bf260f823498 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:17.952001 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1334/2658] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:38:17 2022] PUT /identity/v3/projects/2b095e38cd4b4f3c9e8c3ab2dbd4bb90/users/f0115f43efba4e5ba90b4b8a9d8e4a21/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:17.952726 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8ce26-bb20-4235-ae13-6286adb74902 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:17.964716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8ce26-bb20-4235-ae13-6286adb74902 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:18.035644 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b700109-120f-493a-b44f-5d7bf00c7fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:18.037051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b700109-120f-493a-b44f-5d7bf00c7fec admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:18.037153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b700109-120f-493a-b44f-5d7bf00c7fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:18.037548 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b700109-120f-493a-b44f-5d7bf00c7fec admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:18.038265 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b700109-120f-493a-b44f-5d7bf00c7fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:18.040255 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b700109-120f-493a-b44f-5d7bf00c7fec admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:18.053379 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1325/2659] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:38:18.058143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddf476-105d-42f3-a19d-bf260f823498 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:18.074806 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddf476-105d-42f3-a19d-bf260f823498 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:18.096412 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aadc1a-cf3e-454e-b95c-36cf695b9362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:18.097554 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aadc1a-cf3e-454e-b95c-36cf695b9362 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:18.097941 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aadc1a-cf3e-454e-b95c-36cf695b9362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:18.099277 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aadc1a-cf3e-454e-b95c-36cf695b9362 admin admin] PATH_INFO: `/v3/projects/2b095e38cd4b4f3c9e8c3ab2dbd4bb90/users/f0115f43efba4e5ba90b4b8a9d8e4a21/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:18.101288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aadc1a-cf3e-454e-b95c-36cf695b9362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b095e38cd4b4f3c9e8c3ab2dbd4bb90, user_id=f0115f43efba4e5ba90b4b8a9d8e4a21, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:18.104021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aadc1a-cf3e-454e-b95c-36cf695b9362 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:18.125660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1335/2660] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:38:18 2022] PUT /identity/v3/projects/2b095e38cd4b4f3c9e8c3ab2dbd4bb90/users/f0115f43efba4e5ba90b4b8a9d8e4a21/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:18.196745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0542cf9-7d73-485d-9e9a-3deb311c9b2c None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:18.197031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0542cf9-7d73-485d-9e9a-3deb311c9b2c None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:18.197305 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0542cf9-7d73-485d-9e9a-3deb311c9b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:18.234306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-c0542cf9-7d73-485d-9e9a-3deb311c9b2c None None] MFA Rules not processed for user `f0115f43efba4e5ba90b4b8a9d8e4a21`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:18.235646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c0542cf9-7d73-485d-9e9a-3deb311c9b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:18.326663 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1326/2661] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:18.339517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aadc1a-cf3e-454e-b95c-36cf695b9362 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:18.341294 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c1aadc1a-cf3e-454e-b95c-36cf695b9362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:18.411887 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aadc1a-cf3e-454e-b95c-36cf695b9362 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:18.415884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17ebca-f88e-4737-a586-bd23df848852 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:18.416637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17ebca-f88e-4737-a586-bd23df848852 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:18.417281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17ebca-f88e-4737-a586-bd23df848852 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:18.417652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17ebca-f88e-4737-a586-bd23df848852 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:18.428137 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17ebca-f88e-4737-a586-bd23df848852 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:18.429663 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17ebca-f88e-4737-a586-bd23df848852 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:18.441705 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1336/2662] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:38:18.654705 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0542cf9-7d73-485d-9e9a-3deb311c9b2c None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:18.667154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0542cf9-7d73-485d-9e9a-3deb311c9b2c None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:18.671557 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd2db4-33fe-4cd2-9db7-ce0035fc383c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:18.672276 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd2db4-33fe-4cd2-9db7-ce0035fc383c service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:18.672634 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd2db4-33fe-4cd2-9db7-ce0035fc383c service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:18.673044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd2db4-33fe-4cd2-9db7-ce0035fc383c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:18.737426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:18.752692 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd2db4-33fe-4cd2-9db7-ce0035fc383c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:18.755039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd2db4-33fe-4cd2-9db7-ce0035fc383c service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:18.768514 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1327/2663] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:38:27.031375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17ebca-f88e-4737-a586-bd23df848852 service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:27.041837 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:27.148261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17ebca-f88e-4737-a586-bd23df848852 service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:27.155202 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899f955-b4a0-4ca7-b86e-4f0107527764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:27.156009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899f955-b4a0-4ca7-b86e-4f0107527764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:27.156409 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899f955-b4a0-4ca7-b86e-4f0107527764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:27.156737 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899f955-b4a0-4ca7-b86e-4f0107527764 admin admin] PATH_INFO: `/v3/users/3a71a5ff3e404920bda6e5affc0add63` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:27.157932 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899f955-b4a0-4ca7-b86e-4f0107527764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a71a5ff3e404920bda6e5affc0add63)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:27.159935 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899f955-b4a0-4ca7-b86e-4f0107527764 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:27.273278 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce1175-dd78-4bfe-bfb3-30392d9e6773 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:27.273855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce1175-dd78-4bfe-bfb3-30392d9e6773 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:27.273855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce1175-dd78-4bfe-bfb3-30392d9e6773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:27.297411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-a6ce1175-dd78-4bfe-bfb3-30392d9e6773 None None] MFA Rules not processed for user `991f6d49ad014318bc9c2f5d57b9deb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:27.298638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a6ce1175-dd78-4bfe-bfb3-30392d9e6773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:27.355768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1328/2664] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:38:27.661245 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c899f955-b4a0-4ca7-b86e-4f0107527764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a71a5ff3e404920bda6e5affc0add63'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:27.688492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c899f955-b4a0-4ca7-b86e-4f0107527764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a71a5ff3e404920bda6e5affc0add63'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:27.718015 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce1175-dd78-4bfe-bfb3-30392d9e6773 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:27.719551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a6ce1175-dd78-4bfe-bfb3-30392d9e6773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:27.757829 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c899f955-b4a0-4ca7-b86e-4f0107527764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a71a5ff3e404920bda6e5affc0add63'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:27.771825 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c899f955-b4a0-4ca7-b86e-4f0107527764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a71a5ff3e404920bda6e5affc0add63'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:27.773637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c899f955-b4a0-4ca7-b86e-4f0107527764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a71a5ff3e404920bda6e5affc0add63'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:27.784078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1337/2665] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:38:27 2022] DELETE /identity/v3/users/3a71a5ff3e404920bda6e5affc0add63 => generated 0 bytes in 753 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:27.791399 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce1175-dd78-4bfe-bfb3-30392d9e6773 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:27.792714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00280a-61f2-4696-b3d2-a80b29845040 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:27.793055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00280a-61f2-4696-b3d2-a80b29845040 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:27.793363 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00280a-61f2-4696-b3d2-a80b29845040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:27.822607 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-6c00280a-61f2-4696-b3d2-a80b29845040 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:27.824052 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6c00280a-61f2-4696-b3d2-a80b29845040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:27.866595 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cfe32-50ef-414c-815f-d6fbfc5f3d49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:27.867890 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cfe32-50ef-414c-815f-d6fbfc5f3d49 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:27.868654 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cfe32-50ef-414c-815f-d6fbfc5f3d49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:27.869193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cfe32-50ef-414c-815f-d6fbfc5f3d49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:27.872525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-015cfe32-50ef-414c-815f-d6fbfc5f3d49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:27.889104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cfe32-50ef-414c-815f-d6fbfc5f3d49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:27.891941 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cfe32-50ef-414c-815f-d6fbfc5f3d49 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:27.908087 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1338/2666] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:27.909696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1329/2667] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:38:27.919643 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00280a-61f2-4696-b3d2-a80b29845040 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:27.921241 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6c00280a-61f2-4696-b3d2-a80b29845040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:27.939579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00280a-61f2-4696-b3d2-a80b29845040 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:27.945803 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab90e4-be37-4720-aa37-19350b8f31b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:27.946575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab90e4-be37-4720-aa37-19350b8f31b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:27.947036 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab90e4-be37-4720-aa37-19350b8f31b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:27.947445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab90e4-be37-4720-aa37-19350b8f31b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:27.962087 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab90e4-be37-4720-aa37-19350b8f31b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:27.963895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab90e4-be37-4720-aa37-19350b8f31b4 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:27.976204 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1339/2668] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:38:28.509118 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cfe32-50ef-414c-815f-d6fbfc5f3d49 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:28.511614 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-015cfe32-50ef-414c-815f-d6fbfc5f3d49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:28.534386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cfe32-50ef-414c-815f-d6fbfc5f3d49 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:28.541227 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:28.541928 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:28.542313 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:28.542851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6 admin admin] PATH_INFO: `/v3/projects/9579dcaed8eb4e79988b7e5e11e272b6` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:28.544058 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9579dcaed8eb4e79988b7e5e11e272b6)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:28.546438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:28.688895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:28.691691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9579dcaed8eb4e79988b7e5e11e272b6', 'request_id': 'req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:28.697193 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9579dcaed8eb4e79988b7e5e11e272b6', 'request_id': 'req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:28.703368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9579dcaed8eb4e79988b7e5e11e272b6', 'request_id': 'req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:28.803491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6 admin admin] The token cache is being invalidate because project 9579dcaed8eb4e79988b7e5e11e272b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:38:28.803787 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:28.807882 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:28.807882 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1330/2669] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:38:28 2022] DELETE /identity/v3/projects/9579dcaed8eb4e79988b7e5e11e272b6 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:28.810028 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab90e4-be37-4720-aa37-19350b8f31b4 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:28.821392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab90e4-be37-4720-aa37-19350b8f31b4 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:28.833299 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:28.885057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f149fe-d453-42bc-aa51-b28b718d59db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:28.885716 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f149fe-d453-42bc-aa51-b28b718d59db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:28.886008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f149fe-d453-42bc-aa51-b28b718d59db admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:28.886263 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f149fe-d453-42bc-aa51-b28b718d59db admin admin] PATH_INFO: `/v3/users/f0115f43efba4e5ba90b4b8a9d8e4a21` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:28.887540 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f149fe-d453-42bc-aa51-b28b718d59db admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0115f43efba4e5ba90b4b8a9d8e4a21)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:28.889423 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f149fe-d453-42bc-aa51-b28b718d59db admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:29.040438 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-04f149fe-d453-42bc-aa51-b28b718d59db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0115f43efba4e5ba90b4b8a9d8e4a21'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:29.052623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-04f149fe-d453-42bc-aa51-b28b718d59db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0115f43efba4e5ba90b4b8a9d8e4a21'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:29.135120 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-04f149fe-d453-42bc-aa51-b28b718d59db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0115f43efba4e5ba90b4b8a9d8e4a21'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:29.143902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-04f149fe-d453-42bc-aa51-b28b718d59db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0115f43efba4e5ba90b4b8a9d8e4a21'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:29.146336 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-04f149fe-d453-42bc-aa51-b28b718d59db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0115f43efba4e5ba90b4b8a9d8e4a21'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:29.153293 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1340/2670] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:38:28 2022] DELETE /identity/v3/users/f0115f43efba4e5ba90b4b8a9d8e4a21 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:29.624440 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:29.626573 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:29.644067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e72a0-159d-4ca5-a80f-a1c4662a4da6 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:29.708398 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd6fa5-f590-4bd3-80b1-2dda47e48aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:29.709048 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd6fa5-f590-4bd3-80b1-2dda47e48aab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:29.709322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd6fa5-f590-4bd3-80b1-2dda47e48aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:29.709590 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd6fa5-f590-4bd3-80b1-2dda47e48aab admin admin] PATH_INFO: `/v3/projects/2b095e38cd4b4f3c9e8c3ab2dbd4bb90` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:29.710773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd6fa5-f590-4bd3-80b1-2dda47e48aab admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b095e38cd4b4f3c9e8c3ab2dbd4bb90)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:29.712708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd6fa5-f590-4bd3-80b1-2dda47e48aab admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:29.815248 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-17fd6fa5-f590-4bd3-80b1-2dda47e48aab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b095e38cd4b4f3c9e8c3ab2dbd4bb90', 'request_id': 'req-17fd6fa5-f590-4bd3-80b1-2dda47e48aab'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:29.821898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-17fd6fa5-f590-4bd3-80b1-2dda47e48aab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b095e38cd4b4f3c9e8c3ab2dbd4bb90', 'request_id': 'req-17fd6fa5-f590-4bd3-80b1-2dda47e48aab'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:29.828437 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-17fd6fa5-f590-4bd3-80b1-2dda47e48aab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b095e38cd4b4f3c9e8c3ab2dbd4bb90', 'request_id': 'req-17fd6fa5-f590-4bd3-80b1-2dda47e48aab'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:29.929357 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-17fd6fa5-f590-4bd3-80b1-2dda47e48aab admin admin] The token cache is being invalidate because project 2b095e38cd4b4f3c9e8c3ab2dbd4bb90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:38:29.929658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-17fd6fa5-f590-4bd3-80b1-2dda47e48aab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:29.930290 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-17fd6fa5-f590-4bd3-80b1-2dda47e48aab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:29.931840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1331/2671] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:38:29 2022] DELETE /identity/v3/projects/2b095e38cd4b4f3c9e8c3ab2dbd4bb90 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:29.937100 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe63130-d44b-439a-a97b-b74914ed1e15 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:29.937899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe63130-d44b-439a-a97b-b74914ed1e15 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:29.938072 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe63130-d44b-439a-a97b-b74914ed1e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:29.959484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-4fe63130-d44b-439a-a97b-b74914ed1e15 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:29.962539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4fe63130-d44b-439a-a97b-b74914ed1e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:30.021979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1341/2672] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:30.029808 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da6ff3-ea64-471e-94a6-e722bf3e0c07 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:30.030120 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da6ff3-ea64-471e-94a6-e722bf3e0c07 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:30.031355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da6ff3-ea64-471e-94a6-e722bf3e0c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:30.051185 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-79da6ff3-ea64-471e-94a6-e722bf3e0c07 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:30.053274 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-79da6ff3-ea64-471e-94a6-e722bf3e0c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:30.061566 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1332/2673] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:38:30.067561 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe63130-d44b-439a-a97b-b74914ed1e15 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:30.077931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe63130-d44b-439a-a97b-b74914ed1e15 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:30.083581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ae552-20ad-426a-8876-9ff03ebaf712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:30.084202 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ae552-20ad-426a-8876-9ff03ebaf712 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:30.084483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ae552-20ad-426a-8876-9ff03ebaf712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:30.084759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ae552-20ad-426a-8876-9ff03ebaf712 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:30.085415 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ae552-20ad-426a-8876-9ff03ebaf712 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:38:30.085722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ae552-20ad-426a-8876-9ff03ebaf712 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:30.087434 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ae552-20ad-426a-8876-9ff03ebaf712 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:30.103761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1342/2674] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:38:30.104849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da6ff3-ea64-471e-94a6-e722bf3e0c07 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:30.115551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da6ff3-ea64-471e-94a6-e722bf3e0c07 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:30.120301 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8ce6b-b58e-4545-b410-2549ea4fc12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:30.120900 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8ce6b-b58e-4545-b410-2549ea4fc12f admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:30.121178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8ce6b-b58e-4545-b410-2549ea4fc12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:30.121450 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8ce6b-b58e-4545-b410-2549ea4fc12f admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:30.122073 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8ce6b-b58e-4545-b410-2549ea4fc12f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:30.124042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8ce6b-b58e-4545-b410-2549ea4fc12f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:30.223345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1333/2675] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:38:30 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:30.228151 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ae552-20ad-426a-8876-9ff03ebaf712 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:30.240049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ae552-20ad-426a-8876-9ff03ebaf712 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:30.260093 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:30.305878 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6e1bd-e29a-4827-9896-4945847a181c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:30.306808 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6e1bd-e29a-4827-9896-4945847a181c admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:30.307501 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6e1bd-e29a-4827-9896-4945847a181c admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:30.307993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6e1bd-e29a-4827-9896-4945847a181c admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:30.308702 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6e1bd-e29a-4827-9896-4945847a181c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:30.311004 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6e1bd-e29a-4827-9896-4945847a181c admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:30.317751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6a81e-539c-48c4-9dbc-58ba440c11e1 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:30.318142 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6a81e-539c-48c4-9dbc-58ba440c11e1 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:30.318483 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6a81e-539c-48c4-9dbc-58ba440c11e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:30.338434 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-45a6a81e-539c-48c4-9dbc-58ba440c11e1 None None] MFA Rules not processed for user `1ee990b1658e49cc8bc2fe15e7dc36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:30.339674 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-45a6a81e-539c-48c4-9dbc-58ba440c11e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:30.370308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:30.396223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1334/2676] 10.222.0.59 () {58 vars in 967 bytes} [Wed Aug 31 19:38:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:38:30.408422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6a81e-539c-48c4-9dbc-58ba440c11e1 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:30.409800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-45a6a81e-539c-48c4-9dbc-58ba440c11e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:30.416843 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1343/2677] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:38:30 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:30.422022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6e1bd-e29a-4827-9896-4945847a181c admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:30.432796 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6e1bd-e29a-4827-9896-4945847a181c admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:30.438034 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a8fda-374d-4746-9f4b-ff3954ff91c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:30.443685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a8fda-374d-4746-9f4b-ff3954ff91c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:30.443685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a8fda-374d-4746-9f4b-ff3954ff91c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:30.443685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a8fda-374d-4746-9f4b-ff3954ff91c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:30.443685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a8fda-374d-4746-9f4b-ff3954ff91c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:30.443685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a8fda-374d-4746-9f4b-ff3954ff91c1 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:30.453483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1344/2678] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:38:30.457622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a8fda-374d-4746-9f4b-ff3954ff91c1 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:30.468096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a8fda-374d-4746-9f4b-ff3954ff91c1 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:30.472620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcf9c8-2a16-4d9c-8974-6bdc08d3fc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:30.473220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcf9c8-2a16-4d9c-8974-6bdc08d3fc6f admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:30.474231 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcf9c8-2a16-4d9c-8974-6bdc08d3fc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:30.474578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcf9c8-2a16-4d9c-8974-6bdc08d3fc6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:30.475394 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcf9c8-2a16-4d9c-8974-6bdc08d3fc6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:30.476990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcf9c8-2a16-4d9c-8974-6bdc08d3fc6f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:30.485013 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6a81e-539c-48c4-9dbc-58ba440c11e1 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:30.490273 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1345/2679] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:38:30.491351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcf9c8-2a16-4d9c-8974-6bdc08d3fc6f admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:30.492588 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79dbe50-8b26-4e3a-9ac6-a05f65329387 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25ed58369ffd46e8b7fa1880662926e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:30.493306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dbe50-8b26-4e3a-9ac6-a05f65329387 service swift] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:30.493591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dbe50-8b26-4e3a-9ac6-a05f65329387 service swift] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:30.493869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dbe50-8b26-4e3a-9ac6-a05f65329387 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:30.501461 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcf9c8-2a16-4d9c-8974-6bdc08d3fc6f admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:30.502998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79dbe50-8b26-4e3a-9ac6-a05f65329387 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:30.504694 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79dbe50-8b26-4e3a-9ac6-a05f65329387 service swift] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:30.506356 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480adaa-e11e-4459-a544-f3fc27f875b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:30.507066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480adaa-e11e-4459-a544-f3fc27f875b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:30.507346 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480adaa-e11e-4459-a544-f3fc27f875b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:30.507618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480adaa-e11e-4459-a544-f3fc27f875b3 admin admin] PATH_INFO: `/v3/projects/65e216d4b3fe41bc94db8f74c1fdcecd/users/fbe9fb211163474090507b81a1016415/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:30.515477 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1335/2680] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 31 19:38:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:38:30.523643 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480adaa-e11e-4459-a544-f3fc27f875b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65e216d4b3fe41bc94db8f74c1fdcecd, user_id=fbe9fb211163474090507b81a1016415, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:30.526321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480adaa-e11e-4459-a544-f3fc27f875b3 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:30.574813 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1346/2681] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:38:30 2022] PUT /identity/v3/projects/65e216d4b3fe41bc94db8f74c1fdcecd/users/fbe9fb211163474090507b81a1016415/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:38:30.587309 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826246c6-1ba5-4b0f-816e-373aedd11516 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:30.587592 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826246c6-1ba5-4b0f-816e-373aedd11516 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:30.587870 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826246c6-1ba5-4b0f-816e-373aedd11516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:30.618293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-826246c6-1ba5-4b0f-816e-373aedd11516 None None] MFA Rules not processed for user `fbe9fb211163474090507b81a1016415`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:30.619624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-826246c6-1ba5-4b0f-816e-373aedd11516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:30.701746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1336/2682] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:30.713459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480adaa-e11e-4459-a544-f3fc27f875b3 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:30.716214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3480adaa-e11e-4459-a544-f3fc27f875b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:30.796384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480adaa-e11e-4459-a544-f3fc27f875b3 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:30.800145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23b0b2-8915-4118-a5f1-c3de4f6344c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:30.800756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23b0b2-8915-4118-a5f1-c3de4f6344c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:30.801038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23b0b2-8915-4118-a5f1-c3de4f6344c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:30.801331 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23b0b2-8915-4118-a5f1-c3de4f6344c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:30.811536 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23b0b2-8915-4118-a5f1-c3de4f6344c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:30.812977 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23b0b2-8915-4118-a5f1-c3de4f6344c6 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:30.822301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1347/2683] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:38:33.765039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826246c6-1ba5-4b0f-816e-373aedd11516 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:33.766446 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-826246c6-1ba5-4b0f-816e-373aedd11516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:33.835802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=100 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:33.842135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826246c6-1ba5-4b0f-816e-373aedd11516 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:33.847232 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c881cd-5147-4f54-aefa-f5254bef825f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:33.847843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c881cd-5147-4f54-aefa-f5254bef825f service placement] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:33.848132 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c881cd-5147-4f54-aefa-f5254bef825f service placement] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:33.848416 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c881cd-5147-4f54-aefa-f5254bef825f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:33.850138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-32c881cd-5147-4f54-aefa-f5254bef825f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:33.862442 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c881cd-5147-4f54-aefa-f5254bef825f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:33.864068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c881cd-5147-4f54-aefa-f5254bef825f service placement] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:33.874965 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1337/2684] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:38:33.906689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23b0b2-8915-4118-a5f1-c3de4f6344c6 service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:33.908281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-fe23b0b2-8915-4118-a5f1-c3de4f6344c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:33.984274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23b0b2-8915-4118-a5f1-c3de4f6344c6 service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:33.989061 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fb45c-4c8b-4d53-80d8-6f48e70d887b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:33.989698 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fb45c-4c8b-4d53-80d8-6f48e70d887b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:33.989989 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fb45c-4c8b-4d53-80d8-6f48e70d887b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:33.990695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fb45c-4c8b-4d53-80d8-6f48e70d887b admin admin] PATH_INFO: `/v3/users/1d68da36768d48a2bccb0fd2fd1a7acb` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:33.992274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fb45c-4c8b-4d53-80d8-6f48e70d887b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d68da36768d48a2bccb0fd2fd1a7acb)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:33.994787 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fb45c-4c8b-4d53-80d8-6f48e70d887b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:34.000750 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:34.177467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-5b8fb45c-4c8b-4d53-80d8-6f48e70d887b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d68da36768d48a2bccb0fd2fd1a7acb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:34.190138 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-5b8fb45c-4c8b-4d53-80d8-6f48e70d887b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d68da36768d48a2bccb0fd2fd1a7acb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:34.277383 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-5b8fb45c-4c8b-4d53-80d8-6f48e70d887b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d68da36768d48a2bccb0fd2fd1a7acb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:34.285065 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-5b8fb45c-4c8b-4d53-80d8-6f48e70d887b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d68da36768d48a2bccb0fd2fd1a7acb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:34.285748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-5b8fb45c-4c8b-4d53-80d8-6f48e70d887b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d68da36768d48a2bccb0fd2fd1a7acb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:34.292729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1348/2685] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:38:33 2022] DELETE /identity/v3/users/1d68da36768d48a2bccb0fd2fd1a7acb => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:34.570786 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c881cd-5147-4f54-aefa-f5254bef825f service placement] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:34.581960 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c881cd-5147-4f54-aefa-f5254bef825f service placement] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:34.644030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f3e2e-0f2c-43a2-9279-9d35139dfd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:34.644737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f3e2e-0f2c-43a2-9279-9d35139dfd31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:34.645022 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f3e2e-0f2c-43a2-9279-9d35139dfd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:34.645297 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f3e2e-0f2c-43a2-9279-9d35139dfd31 admin admin] PATH_INFO: `/v3/projects/b8270db003fa4442afb38ba99ee0263e` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:34.646423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f3e2e-0f2c-43a2-9279-9d35139dfd31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8270db003fa4442afb38ba99ee0263e)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:34.648997 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f3e2e-0f2c-43a2-9279-9d35139dfd31 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:34.757156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-5e1f3e2e-0f2c-43a2-9279-9d35139dfd31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8270db003fa4442afb38ba99ee0263e', 'request_id': 'req-5e1f3e2e-0f2c-43a2-9279-9d35139dfd31'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:34.763300 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-5e1f3e2e-0f2c-43a2-9279-9d35139dfd31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8270db003fa4442afb38ba99ee0263e', 'request_id': 'req-5e1f3e2e-0f2c-43a2-9279-9d35139dfd31'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:34.769540 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-5e1f3e2e-0f2c-43a2-9279-9d35139dfd31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8270db003fa4442afb38ba99ee0263e', 'request_id': 'req-5e1f3e2e-0f2c-43a2-9279-9d35139dfd31'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:34.856033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-5e1f3e2e-0f2c-43a2-9279-9d35139dfd31 admin admin] The token cache is being invalidate because project b8270db003fa4442afb38ba99ee0263e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:38:34.856534 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-5e1f3e2e-0f2c-43a2-9279-9d35139dfd31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:34.857443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-5e1f3e2e-0f2c-43a2-9279-9d35139dfd31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:34.859118 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1338/2686] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:38:34 2022] DELETE /identity/v3/projects/b8270db003fa4442afb38ba99ee0263e => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:34.864065 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4625fd1-a8df-45a0-9a10-c89f0a0422b0 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:34.864351 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4625fd1-a8df-45a0-9a10-c89f0a0422b0 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:34.864628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4625fd1-a8df-45a0-9a10-c89f0a0422b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:34.887218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-d4625fd1-a8df-45a0-9a10-c89f0a0422b0 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:34.888400 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d4625fd1-a8df-45a0-9a10-c89f0a0422b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:35.078143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1349/2687] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:35.086856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1c8b7-f05a-4180-9aec-6189bd663286 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:35.087248 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1c8b7-f05a-4180-9aec-6189bd663286 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:35.087524 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1c8b7-f05a-4180-9aec-6189bd663286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:35.107954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-90a1c8b7-f05a-4180-9aec-6189bd663286 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:35.109150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-90a1c8b7-f05a-4180-9aec-6189bd663286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:35.116784 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1339/2688] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:35.122219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4625fd1-a8df-45a0-9a10-c89f0a0422b0 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:35.131873 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4625fd1-a8df-45a0-9a10-c89f0a0422b0 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:35.137257 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c23d89-b5e1-4adf-8702-8a3ebd7ffd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:35.137853 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c23d89-b5e1-4adf-8702-8a3ebd7ffd02 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:35.138133 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c23d89-b5e1-4adf-8702-8a3ebd7ffd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:35.138405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c23d89-b5e1-4adf-8702-8a3ebd7ffd02 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:35.139123 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c23d89-b5e1-4adf-8702-8a3ebd7ffd02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:38:35.141033 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c23d89-b5e1-4adf-8702-8a3ebd7ffd02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:35.141033 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c23d89-b5e1-4adf-8702-8a3ebd7ffd02 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:35.153631 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1350/2689] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:38:35.157957 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1c8b7-f05a-4180-9aec-6189bd663286 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:35.167595 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1c8b7-f05a-4180-9aec-6189bd663286 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:35.172254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff991238-38ed-4808-9fd5-9f358dad5146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:35.172851 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff991238-38ed-4808-9fd5-9f358dad5146 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:35.173128 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff991238-38ed-4808-9fd5-9f358dad5146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:35.173426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff991238-38ed-4808-9fd5-9f358dad5146 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:35.174050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff991238-38ed-4808-9fd5-9f358dad5146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:35.175965 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff991238-38ed-4808-9fd5-9f358dad5146 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:35.267037 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1340/2690] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:38:35 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:35.271482 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c23d89-b5e1-4adf-8702-8a3ebd7ffd02 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:35.281793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c23d89-b5e1-4adf-8702-8a3ebd7ffd02 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:35.340069 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976fc32-c004-46e5-af84-6dca96bbc980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:35.340590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976fc32-c004-46e5-af84-6dca96bbc980 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:35.340872 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976fc32-c004-46e5-af84-6dca96bbc980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:35.341143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976fc32-c004-46e5-af84-6dca96bbc980 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:35.341767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976fc32-c004-46e5-af84-6dca96bbc980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:35.343792 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976fc32-c004-46e5-af84-6dca96bbc980 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:35.437595 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1351/2691] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:38:35 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:35.441912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff991238-38ed-4808-9fd5-9f358dad5146 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:35.452699 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff991238-38ed-4808-9fd5-9f358dad5146 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:35.457652 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f59ae-6b01-4135-9da8-1976d827338c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:35.458249 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f59ae-6b01-4135-9da8-1976d827338c admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:35.458531 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f59ae-6b01-4135-9da8-1976d827338c admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:35.458805 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f59ae-6b01-4135-9da8-1976d827338c admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:35.459474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f59ae-6b01-4135-9da8-1976d827338c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:35.461403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f59ae-6b01-4135-9da8-1976d827338c admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:35.471894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1341/2692] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:35.475794 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976fc32-c004-46e5-af84-6dca96bbc980 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:35.481682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:35.487491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976fc32-c004-46e5-af84-6dca96bbc980 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:35.492767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc87944-fa3d-4672-95ac-f2b6150cf32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:35.493733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc87944-fa3d-4672-95ac-f2b6150cf32d admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:35.494020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc87944-fa3d-4672-95ac-f2b6150cf32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:35.494292 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc87944-fa3d-4672-95ac-f2b6150cf32d admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:35.495048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc87944-fa3d-4672-95ac-f2b6150cf32d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:35.496647 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc87944-fa3d-4672-95ac-f2b6150cf32d admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:35.510254 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1352/2693] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:35.514435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f59ae-6b01-4135-9da8-1976d827338c admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:35.526338 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f59ae-6b01-4135-9da8-1976d827338c admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:35.531212 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c563c-af95-44c7-aa81-f1c8860067e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:35.531864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c563c-af95-44c7-aa81-f1c8860067e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:35.532144 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c563c-af95-44c7-aa81-f1c8860067e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:35.532416 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c563c-af95-44c7-aa81-f1c8860067e0 admin admin] PATH_INFO: `/v3/projects/a091bdeaca5f4d4e816ce658b3df6ed4/users/f5dfab772b684183bcc244495f6b9f16/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:35.548456 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c563c-af95-44c7-aa81-f1c8860067e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a091bdeaca5f4d4e816ce658b3df6ed4, user_id=f5dfab772b684183bcc244495f6b9f16, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:35.551324 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c563c-af95-44c7-aa81-f1c8860067e0 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:35.644118 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1342/2694] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:38:35 2022] PUT /identity/v3/projects/a091bdeaca5f4d4e816ce658b3df6ed4/users/f5dfab772b684183bcc244495f6b9f16/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:35.716932 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703e8d9-d2b7-4de9-9cad-34c14b10a398 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:35.717279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703e8d9-d2b7-4de9-9cad-34c14b10a398 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:35.717561 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703e8d9-d2b7-4de9-9cad-34c14b10a398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:35.748172 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-8703e8d9-d2b7-4de9-9cad-34c14b10a398 None None] MFA Rules not processed for user `f5dfab772b684183bcc244495f6b9f16`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:35.749389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8703e8d9-d2b7-4de9-9cad-34c14b10a398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:35.814261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1353/2695] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:35.825751 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c563c-af95-44c7-aa81-f1c8860067e0 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:35.827547 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b52c563c-af95-44c7-aa81-f1c8860067e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:35.847900 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:35.908247 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c563c-af95-44c7-aa81-f1c8860067e0 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:35.912639 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256d2db-929a-43e5-93cc-2e934d68462f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f281d778f1404984b4d778a1c26aef1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:35.913252 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256d2db-929a-43e5-93cc-2e934d68462f service glance] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:35.913535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256d2db-929a-43e5-93cc-2e934d68462f service glance] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:35.913816 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256d2db-929a-43e5-93cc-2e934d68462f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:35.924090 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256d2db-929a-43e5-93cc-2e934d68462f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:35.925513 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256d2db-929a-43e5-93cc-2e934d68462f service glance] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:35.934460 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1343/2696] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:38:42.604402 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703e8d9-d2b7-4de9-9cad-34c14b10a398 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:42.607095 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8703e8d9-d2b7-4de9-9cad-34c14b10a398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:42.685973 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703e8d9-d2b7-4de9-9cad-34c14b10a398 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:42.691102 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2541ce-11e1-48c8-9786-ffd89fec3649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:42.691851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2541ce-11e1-48c8-9786-ffd89fec3649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:42.692561 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2541ce-11e1-48c8-9786-ffd89fec3649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:42.693103 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2541ce-11e1-48c8-9786-ffd89fec3649 admin admin] PATH_INFO: `/v3/users/fbe9fb211163474090507b81a1016415` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:42.694368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2541ce-11e1-48c8-9786-ffd89fec3649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbe9fb211163474090507b81a1016415)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:42.696441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2541ce-11e1-48c8-9786-ffd89fec3649 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:42.884793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7c2541ce-11e1-48c8-9786-ffd89fec3649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbe9fb211163474090507b81a1016415'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:42.897483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7c2541ce-11e1-48c8-9786-ffd89fec3649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbe9fb211163474090507b81a1016415'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:42.957011 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7c2541ce-11e1-48c8-9786-ffd89fec3649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbe9fb211163474090507b81a1016415'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:42.964765 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7c2541ce-11e1-48c8-9786-ffd89fec3649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbe9fb211163474090507b81a1016415'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:42.965410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-7c2541ce-11e1-48c8-9786-ffd89fec3649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbe9fb211163474090507b81a1016415'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:42.968339 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:42.974430 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1354/2697] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:38:42 2022] DELETE /identity/v3/users/fbe9fb211163474090507b81a1016415 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:42.979595 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0604c96-3541-434d-9609-afc69af6b353 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:42.980120 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0604c96-3541-434d-9609-afc69af6b353 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:42.980396 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0604c96-3541-434d-9609-afc69af6b353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:43.004917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-a0604c96-3541-434d-9609-afc69af6b353 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:43.006594 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a0604c96-3541-434d-9609-afc69af6b353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:43.071354 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1344/2698] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:43.081795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2541ce-11e1-48c8-9786-ffd89fec3649 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:43.083670 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7c2541ce-11e1-48c8-9786-ffd89fec3649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:43.144305 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2541ce-11e1-48c8-9786-ffd89fec3649 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:43.148049 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b188a7-87bf-4c7d-ad9c-efec8669f0a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:43.148656 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b188a7-87bf-4c7d-ad9c-efec8669f0a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:43.148944 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b188a7-87bf-4c7d-ad9c-efec8669f0a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:43.149299 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b188a7-87bf-4c7d-ad9c-efec8669f0a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:43.163188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b188a7-87bf-4c7d-ad9c-efec8669f0a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:43.164810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b188a7-87bf-4c7d-ad9c-efec8669f0a5 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:43.177032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1355/2699] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:38:43.553391 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0604c96-3541-434d-9609-afc69af6b353 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:43.559151 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a0604c96-3541-434d-9609-afc69af6b353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:43.568719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0604c96-3541-434d-9609-afc69af6b353 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:43.573534 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fe3a8-a01a-40c5-aa0c-932c7b3ac778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:43.574198 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fe3a8-a01a-40c5-aa0c-932c7b3ac778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:43.574482 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fe3a8-a01a-40c5-aa0c-932c7b3ac778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:43.574759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fe3a8-a01a-40c5-aa0c-932c7b3ac778 admin admin] PATH_INFO: `/v3/projects/65e216d4b3fe41bc94db8f74c1fdcecd` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:43.577539 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9fe3a8-a01a-40c5-aa0c-932c7b3ac778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65e216d4b3fe41bc94db8f74c1fdcecd)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:43.581654 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9fe3a8-a01a-40c5-aa0c-932c7b3ac778 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:43.661525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ad9fe3a8-a01a-40c5-aa0c-932c7b3ac778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65e216d4b3fe41bc94db8f74c1fdcecd', 'request_id': 'req-ad9fe3a8-a01a-40c5-aa0c-932c7b3ac778'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:43.667859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ad9fe3a8-a01a-40c5-aa0c-932c7b3ac778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65e216d4b3fe41bc94db8f74c1fdcecd', 'request_id': 'req-ad9fe3a8-a01a-40c5-aa0c-932c7b3ac778'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:43.675914 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ad9fe3a8-a01a-40c5-aa0c-932c7b3ac778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65e216d4b3fe41bc94db8f74c1fdcecd', 'request_id': 'req-ad9fe3a8-a01a-40c5-aa0c-932c7b3ac778'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:43.739867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ad9fe3a8-a01a-40c5-aa0c-932c7b3ac778 admin admin] The token cache is being invalidate because project 65e216d4b3fe41bc94db8f74c1fdcecd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:38:43.740153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ad9fe3a8-a01a-40c5-aa0c-932c7b3ac778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:43.740768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-ad9fe3a8-a01a-40c5-aa0c-932c7b3ac778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:43.742228 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1345/2700] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:38:43 2022] DELETE /identity/v3/projects/65e216d4b3fe41bc94db8f74c1fdcecd => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:38:43.747323 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc9526-d1f1-4881-ab0b-14ab51a4f1bd None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:43.747923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc9526-d1f1-4881-ab0b-14ab51a4f1bd None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:43.747923 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc9526-d1f1-4881-ab0b-14ab51a4f1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:43.767453 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-17fc9526-d1f1-4881-ab0b-14ab51a4f1bd None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:43.768664 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-17fc9526-d1f1-4881-ab0b-14ab51a4f1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:43.827528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1356/2701] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:43.835045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a71da8-9d74-4e73-9663-e6a3a8678367 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:43.835402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a71da8-9d74-4e73-9663-e6a3a8678367 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:43.835733 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a71da8-9d74-4e73-9663-e6a3a8678367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:43.857403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-62a71da8-9d74-4e73-9663-e6a3a8678367 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:43.858801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-62a71da8-9d74-4e73-9663-e6a3a8678367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:43.867874 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1346/2702] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:43.872121 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc9526-d1f1-4881-ab0b-14ab51a4f1bd None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:43.882449 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc9526-d1f1-4881-ab0b-14ab51a4f1bd None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:43.889191 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f8e93-3452-4aa9-bdef-29de8411f7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:43.889810 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f8e93-3452-4aa9-bdef-29de8411f7c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:43.890089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f8e93-3452-4aa9-bdef-29de8411f7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:43.890357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f8e93-3452-4aa9-bdef-29de8411f7c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:43.891051 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f8e93-3452-4aa9-bdef-29de8411f7c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:38:43.891354 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f8e93-3452-4aa9-bdef-29de8411f7c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:43.892954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f8e93-3452-4aa9-bdef-29de8411f7c8 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:43.905795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1357/2703] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:38:43.917753 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a71da8-9d74-4e73-9663-e6a3a8678367 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:43.928219 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a71da8-9d74-4e73-9663-e6a3a8678367 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:43.932943 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6640f31-311a-43ba-8401-e768a6929974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:43.933578 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6640f31-311a-43ba-8401-e768a6929974 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:43.933856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6640f31-311a-43ba-8401-e768a6929974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:43.934126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6640f31-311a-43ba-8401-e768a6929974 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:43.934825 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6640f31-311a-43ba-8401-e768a6929974 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:43.936765 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6640f31-311a-43ba-8401-e768a6929974 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:43.953709 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:44.022181 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1347/2704] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:38:43 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:44.026972 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f8e93-3452-4aa9-bdef-29de8411f7c8 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:44.038816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f8e93-3452-4aa9-bdef-29de8411f7c8 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:44.111668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e4110-4f75-4705-bce4-e5d18514154c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:44.112199 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e4110-4f75-4705-bce4-e5d18514154c admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:44.112510 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e4110-4f75-4705-bce4-e5d18514154c admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:44.112784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e4110-4f75-4705-bce4-e5d18514154c admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:44.113414 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e4110-4f75-4705-bce4-e5d18514154c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:44.115386 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e4110-4f75-4705-bce4-e5d18514154c admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:44.196867 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1358/2705] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:38:44 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:44.201431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6640f31-311a-43ba-8401-e768a6929974 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:44.213056 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6640f31-311a-43ba-8401-e768a6929974 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:44.218032 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f98a6-a5e9-4e03-afa1-288d054adb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:44.218655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f98a6-a5e9-4e03-afa1-288d054adb19 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:44.218962 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f98a6-a5e9-4e03-afa1-288d054adb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:44.219244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f98a6-a5e9-4e03-afa1-288d054adb19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:44.219894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f98a6-a5e9-4e03-afa1-288d054adb19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:44.221537 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f98a6-a5e9-4e03-afa1-288d054adb19 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:44.234204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1348/2706] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:44.238499 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e4110-4f75-4705-bce4-e5d18514154c admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:44.250006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e4110-4f75-4705-bce4-e5d18514154c admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:44.254666 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af48fd6-7aa8-4755-a946-d36731d36e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:44.255298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af48fd6-7aa8-4755-a946-d36731d36e93 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:44.255580 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af48fd6-7aa8-4755-a946-d36731d36e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:44.255854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af48fd6-7aa8-4755-a946-d36731d36e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:44.256497 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af48fd6-7aa8-4755-a946-d36731d36e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:44.258114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af48fd6-7aa8-4755-a946-d36731d36e93 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:44.268639 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1359/2707] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:44.272947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f98a6-a5e9-4e03-afa1-288d054adb19 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:44.283113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f98a6-a5e9-4e03-afa1-288d054adb19 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:44.287676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f0c60-c9a6-4490-9f3c-89f7b3fefd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:44.288329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f0c60-c9a6-4490-9f3c-89f7b3fefd9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:44.288607 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f0c60-c9a6-4490-9f3c-89f7b3fefd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:44.288880 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f0c60-c9a6-4490-9f3c-89f7b3fefd9e admin admin] PATH_INFO: `/v3/projects/151c6cab447840988fb4e4ad5bad211c/users/32801be79f8a4927ad42315c2280ddda/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:44.304599 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f0c60-c9a6-4490-9f3c-89f7b3fefd9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=151c6cab447840988fb4e4ad5bad211c, user_id=32801be79f8a4927ad42315c2280ddda, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:44.307242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f0c60-c9a6-4490-9f3c-89f7b3fefd9e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:44.392319 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1349/2708] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:38:44 2022] PUT /identity/v3/projects/151c6cab447840988fb4e4ad5bad211c/users/32801be79f8a4927ad42315c2280ddda/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:38:44.402690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137a0d9-15e5-4b6d-bfac-cf33d2f2e2a3 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:44.403012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137a0d9-15e5-4b6d-bfac-cf33d2f2e2a3 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:44.403289 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137a0d9-15e5-4b6d-bfac-cf33d2f2e2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:44.424047 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:44.434640 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-8137a0d9-15e5-4b6d-bfac-cf33d2f2e2a3 None None] MFA Rules not processed for user `32801be79f8a4927ad42315c2280ddda`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:44.436963 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8137a0d9-15e5-4b6d-bfac-cf33d2f2e2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:44.507167 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1360/2709] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:44.511724 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f0c60-c9a6-4490-9f3c-89f7b3fefd9e admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:44.578084 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f0c60-c9a6-4490-9f3c-89f7b3fefd9e admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:44.583784 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d6369-271b-494e-85da-d8efecd32008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:44.587526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d6369-271b-494e-85da-d8efecd32008 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:44.587526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d6369-271b-494e-85da-d8efecd32008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:44.587526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d6369-271b-494e-85da-d8efecd32008 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:44.587526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d6369-271b-494e-85da-d8efecd32008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:44.587526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d6369-271b-494e-85da-d8efecd32008 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:44.650215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1350/2710] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:38:44 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:38:44.661915 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137a0d9-15e5-4b6d-bfac-cf33d2f2e2a3 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:44.676934 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137a0d9-15e5-4b6d-bfac-cf33d2f2e2a3 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:44.739907 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f54d47-cfc9-49de-a432-bf74a7fa4e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:44.740525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f54d47-cfc9-49de-a432-bf74a7fa4e70 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:44.740811 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f54d47-cfc9-49de-a432-bf74a7fa4e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:44.741091 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f54d47-cfc9-49de-a432-bf74a7fa4e70 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:44.741739 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f54d47-cfc9-49de-a432-bf74a7fa4e70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:44.743741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f54d47-cfc9-49de-a432-bf74a7fa4e70 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:44.866703 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1361/2711] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:38:44 2022] POST /identity/v3/users => generated 292 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:44.870846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d6369-271b-494e-85da-d8efecd32008 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:44.883856 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d6369-271b-494e-85da-d8efecd32008 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:44.889454 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2dc76a-938f-4a52-8f05-e5ce770887ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:44.890059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2dc76a-938f-4a52-8f05-e5ce770887ce admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:44.890345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2dc76a-938f-4a52-8f05-e5ce770887ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:44.890624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2dc76a-938f-4a52-8f05-e5ce770887ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:44.891307 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2dc76a-938f-4a52-8f05-e5ce770887ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:44.892900 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2dc76a-938f-4a52-8f05-e5ce770887ce admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:44.904920 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1351/2712] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:44.908781 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f54d47-cfc9-49de-a432-bf74a7fa4e70 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:44.918812 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f54d47-cfc9-49de-a432-bf74a7fa4e70 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:44.923781 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46921e-655e-4943-ba91-72daafd9d410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:44.924414 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46921e-655e-4943-ba91-72daafd9d410 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:44.924704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46921e-655e-4943-ba91-72daafd9d410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:44.924973 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46921e-655e-4943-ba91-72daafd9d410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:44.925610 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46921e-655e-4943-ba91-72daafd9d410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:44.927261 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46921e-655e-4943-ba91-72daafd9d410 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:44.938008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1362/2713] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:44.943125 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2dc76a-938f-4a52-8f05-e5ce770887ce admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:44.953907 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2dc76a-938f-4a52-8f05-e5ce770887ce admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:44.958804 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a44ec-7868-422d-8a5a-5efe84778074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:44.959485 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a44ec-7868-422d-8a5a-5efe84778074 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:44.959763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a44ec-7868-422d-8a5a-5efe84778074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:44.960032 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a44ec-7868-422d-8a5a-5efe84778074 admin admin] PATH_INFO: `/v3/projects/04fee15b6f0c4f3bb76af9c646ef5798/users/a8f594fffee9430e9c9ddf65cf66248d/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:44.978207 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a44ec-7868-422d-8a5a-5efe84778074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04fee15b6f0c4f3bb76af9c646ef5798, user_id=a8f594fffee9430e9c9ddf65cf66248d, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:44.980895 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a44ec-7868-422d-8a5a-5efe84778074 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:45.039702 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1352/2714] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:38:44 2022] PUT /identity/v3/projects/04fee15b6f0c4f3bb76af9c646ef5798/users/a8f594fffee9430e9c9ddf65cf66248d/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:45.052036 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a46a7-681e-4403-9fc7-3e64759739b8 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:45.052330 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a46a7-681e-4403-9fc7-3e64759739b8 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:45.052615 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a46a7-681e-4403-9fc7-3e64759739b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:45.084723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-278a46a7-681e-4403-9fc7-3e64759739b8 None None] MFA Rules not processed for user `a8f594fffee9430e9c9ddf65cf66248d`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:45.085953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-278a46a7-681e-4403-9fc7-3e64759739b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:45.160061 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1363/2715] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:45.164706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a44ec-7868-422d-8a5a-5efe84778074 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:45.236982 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a44ec-7868-422d-8a5a-5efe84778074 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:45.242001 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869498a-bebd-4bc7-900a-b184740f8613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:45.242641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869498a-bebd-4bc7-900a-b184740f8613 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:45.242965 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869498a-bebd-4bc7-900a-b184740f8613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:45.243243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869498a-bebd-4bc7-900a-b184740f8613 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:45.243908 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869498a-bebd-4bc7-900a-b184740f8613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:45.245799 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869498a-bebd-4bc7-900a-b184740f8613 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:45.261822 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:45.324811 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1353/2716] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:38:45 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:45.330046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a46a7-681e-4403-9fc7-3e64759739b8 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:45.340682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a46a7-681e-4403-9fc7-3e64759739b8 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:45.344320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:45.401084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d39d2-6326-4659-90ab-b1e50a84912d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:45.401661 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d39d2-6326-4659-90ab-b1e50a84912d admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:45.401979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d39d2-6326-4659-90ab-b1e50a84912d admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:45.402708 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d39d2-6326-4659-90ab-b1e50a84912d admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:45.404668 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d39d2-6326-4659-90ab-b1e50a84912d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:45.406701 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d39d2-6326-4659-90ab-b1e50a84912d admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:45.511083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1364/2717] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:38:45 2022] POST /identity/v3/users => generated 299 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:45.515882 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869498a-bebd-4bc7-900a-b184740f8613 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:45.526520 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869498a-bebd-4bc7-900a-b184740f8613 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:45.531466 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a32c1-661c-4c44-98c2-15024b3fa028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:45.532089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a32c1-661c-4c44-98c2-15024b3fa028 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:45.532402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a32c1-661c-4c44-98c2-15024b3fa028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:45.532677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a32c1-661c-4c44-98c2-15024b3fa028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:45.533351 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a32c1-661c-4c44-98c2-15024b3fa028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:45.534966 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a32c1-661c-4c44-98c2-15024b3fa028 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:45.546907 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1354/2718] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:45.550493 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d39d2-6326-4659-90ab-b1e50a84912d admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:45.561483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d39d2-6326-4659-90ab-b1e50a84912d admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:45.566189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c8393-074f-4b7c-b597-cbd5cd36b651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:45.566897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c8393-074f-4b7c-b597-cbd5cd36b651 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:45.567180 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c8393-074f-4b7c-b597-cbd5cd36b651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:45.567457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c8393-074f-4b7c-b597-cbd5cd36b651 admin admin] PATH_INFO: `/v3/projects/c7b3761b958a442a96a8e593e783b4bd/users/18d0a059c7b743ab9ff112786d24bda2/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:45.589517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c8393-074f-4b7c-b597-cbd5cd36b651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b3761b958a442a96a8e593e783b4bd, user_id=18d0a059c7b743ab9ff112786d24bda2, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:45.591025 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c8393-074f-4b7c-b597-cbd5cd36b651 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:45.659922 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1365/2719] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:38:45 2022] PUT /identity/v3/projects/c7b3761b958a442a96a8e593e783b4bd/users/18d0a059c7b743ab9ff112786d24bda2/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:45.663779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a32c1-661c-4c44-98c2-15024b3fa028 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:45.674705 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a32c1-661c-4c44-98c2-15024b3fa028 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:45.733281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ef1d3-a695-4131-a76f-cc135845a1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:45.733898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ef1d3-a695-4131-a76f-cc135845a1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:45.734177 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ef1d3-a695-4131-a76f-cc135845a1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:45.734536 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ef1d3-a695-4131-a76f-cc135845a1b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:45.735157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ef1d3-a695-4131-a76f-cc135845a1b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:45.736785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ef1d3-a695-4131-a76f-cc135845a1b5 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:45.749980 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1355/2720] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:45.754161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c8393-074f-4b7c-b597-cbd5cd36b651 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:45.765169 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c8393-074f-4b7c-b597-cbd5cd36b651 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:45.769917 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712e5fc-9303-460a-b13a-30aac8551212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:45.773644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712e5fc-9303-460a-b13a-30aac8551212 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:45.773644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712e5fc-9303-460a-b13a-30aac8551212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:45.773644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712e5fc-9303-460a-b13a-30aac8551212 admin admin] PATH_INFO: `/v3/projects/c7b3761b958a442a96a8e593e783b4bd/users/18d0a059c7b743ab9ff112786d24bda2/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:45.773644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712e5fc-9303-460a-b13a-30aac8551212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b3761b958a442a96a8e593e783b4bd, user_id=18d0a059c7b743ab9ff112786d24bda2, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:45.775823 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712e5fc-9303-460a-b13a-30aac8551212 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:45.786508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1366/2721] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:38:45 2022] PUT /identity/v3/projects/c7b3761b958a442a96a8e593e783b4bd/users/18d0a059c7b743ab9ff112786d24bda2/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:45.797462 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfe93c-9076-4a05-8c95-90774ee09896 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:45.799503 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfe93c-9076-4a05-8c95-90774ee09896 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:45.799503 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfe93c-9076-4a05-8c95-90774ee09896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:45.828706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ecbfe93c-9076-4a05-8c95-90774ee09896 None None] MFA Rules not processed for user `18d0a059c7b743ab9ff112786d24bda2`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:45.830180 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ecbfe93c-9076-4a05-8c95-90774ee09896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:45.898394 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1356/2722] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:45.911212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712e5fc-9303-460a-b13a-30aac8551212 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:45.915713 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-5712e5fc-9303-460a-b13a-30aac8551212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:45.975640 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712e5fc-9303-460a-b13a-30aac8551212 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:45.979337 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de92b0f-9249-434f-9c98-956878c71708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:45.979951 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de92b0f-9249-434f-9c98-956878c71708 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:45.980234 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de92b0f-9249-434f-9c98-956878c71708 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:45.980512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de92b0f-9249-434f-9c98-956878c71708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:46.036240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de92b0f-9249-434f-9c98-956878c71708 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:46.037696 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de92b0f-9249-434f-9c98-956878c71708 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:46.047516 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1367/2723] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:38:49.102565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfe93c-9076-4a05-8c95-90774ee09896 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:49.113958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfe93c-9076-4a05-8c95-90774ee09896 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:49.117742 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e2dcd-c555-43ed-9abb-52c5e32aee93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:49.118362 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e2dcd-c555-43ed-9abb-52c5e32aee93 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:49.118643 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e2dcd-c555-43ed-9abb-52c5e32aee93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:49.118954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e2dcd-c555-43ed-9abb-52c5e32aee93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:49.182000 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e2dcd-c555-43ed-9abb-52c5e32aee93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:49.183474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e2dcd-c555-43ed-9abb-52c5e32aee93 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:49.193074 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1357/2724] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:38:50.674701 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de92b0f-9249-434f-9c98-956878c71708 service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:50.683437 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de92b0f-9249-434f-9c98-956878c71708 service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:50.687581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58535e5e-9f01-4e60-be71-2c283220bc74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:50.688187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58535e5e-9f01-4e60-be71-2c283220bc74 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:50.688467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58535e5e-9f01-4e60-be71-2c283220bc74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:50.688744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58535e5e-9f01-4e60-be71-2c283220bc74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:50.697652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58535e5e-9f01-4e60-be71-2c283220bc74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:50.699097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58535e5e-9f01-4e60-be71-2c283220bc74 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:50.705229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:50.709894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1368/2725] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:38:56.225213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e2dcd-c555-43ed-9abb-52c5e32aee93 service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:56.229645 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:56.290349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e2dcd-c555-43ed-9abb-52c5e32aee93 service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:56.294843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a601b35-467a-4de9-980e-7b08dac365be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:56.295504 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a601b35-467a-4de9-980e-7b08dac365be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:56.295785 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a601b35-467a-4de9-980e-7b08dac365be admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:56.296701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a601b35-467a-4de9-980e-7b08dac365be admin admin] PATH_INFO: `/v3/users/32801be79f8a4927ad42315c2280ddda` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:56.298477 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a601b35-467a-4de9-980e-7b08dac365be admin admin] RBAC: Authorizing `identity:delete_user(user_id=32801be79f8a4927ad42315c2280ddda)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:56.300418 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a601b35-467a-4de9-980e-7b08dac365be admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:56.433704 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6a601b35-467a-4de9-980e-7b08dac365be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32801be79f8a4927ad42315c2280ddda'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:56.482319 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6a601b35-467a-4de9-980e-7b08dac365be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32801be79f8a4927ad42315c2280ddda'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:56.482971 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6a601b35-467a-4de9-980e-7b08dac365be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32801be79f8a4927ad42315c2280ddda'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:56.496536 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6a601b35-467a-4de9-980e-7b08dac365be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32801be79f8a4927ad42315c2280ddda'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:56.497180 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-6a601b35-467a-4de9-980e-7b08dac365be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32801be79f8a4927ad42315c2280ddda'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:56.509568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1358/2726] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:38:56 2022] DELETE /identity/v3/users/32801be79f8a4927ad42315c2280ddda => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:56.514115 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3980ae2-7161-4f56-844d-536850ae4de1 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:56.514644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3980ae2-7161-4f56-844d-536850ae4de1 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:56.514644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3980ae2-7161-4f56-844d-536850ae4de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:56.534657 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a3980ae2-7161-4f56-844d-536850ae4de1 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:56.535834 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a3980ae2-7161-4f56-844d-536850ae4de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:56.595832 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1369/2727] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:56.605605 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a601b35-467a-4de9-980e-7b08dac365be admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:56.607054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-6a601b35-467a-4de9-980e-7b08dac365be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:56.667852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a601b35-467a-4de9-980e-7b08dac365be admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:56.672550 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d40d92-bb6e-45ae-b231-c48d54121e1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:56.673166 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d40d92-bb6e-45ae-b231-c48d54121e1d service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:56.673451 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d40d92-bb6e-45ae-b231-c48d54121e1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:56.673725 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d40d92-bb6e-45ae-b231-c48d54121e1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:56.682883 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d40d92-bb6e-45ae-b231-c48d54121e1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:56.684714 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d40d92-bb6e-45ae-b231-c48d54121e1d service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:56.695268 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1359/2728] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:38:57.078839 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3980ae2-7161-4f56-844d-536850ae4de1 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:57.080576 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a3980ae2-7161-4f56-844d-536850ae4de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:57.094013 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3980ae2-7161-4f56-844d-536850ae4de1 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:57.097850 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c93e20-5f67-4f27-a6fb-1725d74768c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:57.098487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c93e20-5f67-4f27-a6fb-1725d74768c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:57.098761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c93e20-5f67-4f27-a6fb-1725d74768c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:57.099074 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c93e20-5f67-4f27-a6fb-1725d74768c7 admin admin] PATH_INFO: `/v3/projects/151c6cab447840988fb4e4ad5bad211c` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:57.100191 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c93e20-5f67-4f27-a6fb-1725d74768c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=151c6cab447840988fb4e4ad5bad211c)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:57.102086 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c93e20-5f67-4f27-a6fb-1725d74768c7 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:57.192672 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-59c93e20-5f67-4f27-a6fb-1725d74768c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '151c6cab447840988fb4e4ad5bad211c', 'request_id': 'req-59c93e20-5f67-4f27-a6fb-1725d74768c7'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:57.198459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-59c93e20-5f67-4f27-a6fb-1725d74768c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '151c6cab447840988fb4e4ad5bad211c', 'request_id': 'req-59c93e20-5f67-4f27-a6fb-1725d74768c7'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:57.276934 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-59c93e20-5f67-4f27-a6fb-1725d74768c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '151c6cab447840988fb4e4ad5bad211c', 'request_id': 'req-59c93e20-5f67-4f27-a6fb-1725d74768c7'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:57.291396 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-59c93e20-5f67-4f27-a6fb-1725d74768c7 admin admin] The token cache is being invalidate because project 151c6cab447840988fb4e4ad5bad211c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:38:57.291684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-59c93e20-5f67-4f27-a6fb-1725d74768c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:57.292355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-59c93e20-5f67-4f27-a6fb-1725d74768c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:57.293814 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1370/2729] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:38:57 2022] DELETE /identity/v3/projects/151c6cab447840988fb4e4ad5bad211c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:57.298021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d40d92-bb6e-45ae-b231-c48d54121e1d service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:57.307894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d40d92-bb6e-45ae-b231-c48d54121e1d service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:57.329405 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:57.365103 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5de02-74f8-413b-8f88-da426702d499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:57.365721 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5de02-74f8-413b-8f88-da426702d499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:57.365998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5de02-74f8-413b-8f88-da426702d499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:57.366270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5de02-74f8-413b-8f88-da426702d499 admin admin] PATH_INFO: `/v3/users/a8f594fffee9430e9c9ddf65cf66248d` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:57.367546 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a5de02-74f8-413b-8f88-da426702d499 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8f594fffee9430e9c9ddf65cf66248d)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:57.370203 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a5de02-74f8-413b-8f88-da426702d499 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:57.582726 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-76a5de02-74f8-413b-8f88-da426702d499 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8f594fffee9430e9c9ddf65cf66248d'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:57.636513 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-76a5de02-74f8-413b-8f88-da426702d499 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8f594fffee9430e9c9ddf65cf66248d'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:57.637138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-76a5de02-74f8-413b-8f88-da426702d499 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8f594fffee9430e9c9ddf65cf66248d'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:57.649675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-76a5de02-74f8-413b-8f88-da426702d499 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8f594fffee9430e9c9ddf65cf66248d'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:57.650334 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-76a5de02-74f8-413b-8f88-da426702d499 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8f594fffee9430e9c9ddf65cf66248d'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:57.662846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1360/2730] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:38:57 2022] DELETE /identity/v3/users/a8f594fffee9430e9c9ddf65cf66248d => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:58.078748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c93e20-5f67-4f27-a6fb-1725d74768c7 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:58.082565 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-59c93e20-5f67-4f27-a6fb-1725d74768c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:58.094784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c93e20-5f67-4f27-a6fb-1725d74768c7 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:58.150912 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:58.151562 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:58.151847 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:58.152122 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286 admin admin] PATH_INFO: `/v3/projects/04fee15b6f0c4f3bb76af9c646ef5798` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:58.153255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04fee15b6f0c4f3bb76af9c646ef5798)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:58.155756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:58.255672 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:58.272622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04fee15b6f0c4f3bb76af9c646ef5798', 'request_id': 'req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:58.278114 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04fee15b6f0c4f3bb76af9c646ef5798', 'request_id': 'req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:58.368736 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04fee15b6f0c4f3bb76af9c646ef5798', 'request_id': 'req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:58.380222 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286 admin admin] The token cache is being invalidate because project 04fee15b6f0c4f3bb76af9c646ef5798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:38:58.380472 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:58.381120 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:58.382585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1371/2731] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:38:58 2022] DELETE /identity/v3/projects/04fee15b6f0c4f3bb76af9c646ef5798 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:58.386637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5de02-74f8-413b-8f88-da426702d499 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:58.388121 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-76a5de02-74f8-413b-8f88-da426702d499 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:58.401341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5de02-74f8-413b-8f88-da426702d499 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:58.459148 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abd064-d547-4f80-a9a0-6f784fea8d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:58.459864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abd064-d547-4f80-a9a0-6f784fea8d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:58.460137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abd064-d547-4f80-a9a0-6f784fea8d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:58.460407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abd064-d547-4f80-a9a0-6f784fea8d56 admin admin] PATH_INFO: `/v3/users/18d0a059c7b743ab9ff112786d24bda2` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:58.461675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abd064-d547-4f80-a9a0-6f784fea8d56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18d0a059c7b743ab9ff112786d24bda2)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:58.463601 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abd064-d547-4f80-a9a0-6f784fea8d56 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:58.660519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:58.662092 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-41abd064-d547-4f80-a9a0-6f784fea8d56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18d0a059c7b743ab9ff112786d24bda2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:58.756246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-41abd064-d547-4f80-a9a0-6f784fea8d56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18d0a059c7b743ab9ff112786d24bda2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:58.756908 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-41abd064-d547-4f80-a9a0-6f784fea8d56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18d0a059c7b743ab9ff112786d24bda2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:58.772062 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-41abd064-d547-4f80-a9a0-6f784fea8d56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18d0a059c7b743ab9ff112786d24bda2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:58.772757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-41abd064-d547-4f80-a9a0-6f784fea8d56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18d0a059c7b743ab9ff112786d24bda2'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:58.786422 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1361/2732] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:38:58 2022] DELETE /identity/v3/users/18d0a059c7b743ab9ff112786d24bda2 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:59.141921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:59.152655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:59.159256 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a33ac-c2f8-43a3-ac0b-b37e1fd3e286 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:59.219009 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34870286-d7d0-420c-8c5f-78e89cef492c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:59.219568 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34870286-d7d0-420c-8c5f-78e89cef492c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:59.219850 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34870286-d7d0-420c-8c5f-78e89cef492c admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:59.220126 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34870286-d7d0-420c-8c5f-78e89cef492c admin admin] PATH_INFO: `/v3/projects/c7b3761b958a442a96a8e593e783b4bd` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:59.221279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34870286-d7d0-420c-8c5f-78e89cef492c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7b3761b958a442a96a8e593e783b4bd)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:59.223229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34870286-d7d0-420c-8c5f-78e89cef492c admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:59.328832 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-34870286-d7d0-420c-8c5f-78e89cef492c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7b3761b958a442a96a8e593e783b4bd', 'request_id': 'req-34870286-d7d0-420c-8c5f-78e89cef492c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:59.334714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-34870286-d7d0-420c-8c5f-78e89cef492c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7b3761b958a442a96a8e593e783b4bd', 'request_id': 'req-34870286-d7d0-420c-8c5f-78e89cef492c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:59.379446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-34870286-d7d0-420c-8c5f-78e89cef492c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7b3761b958a442a96a8e593e783b4bd', 'request_id': 'req-34870286-d7d0-420c-8c5f-78e89cef492c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:59.391850 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-34870286-d7d0-420c-8c5f-78e89cef492c admin admin] The token cache is being invalidate because project c7b3761b958a442a96a8e593e783b4bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:38:59.393431 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-34870286-d7d0-420c-8c5f-78e89cef492c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:59.393431 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-34870286-d7d0-420c-8c5f-78e89cef492c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:38:59.394399 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1372/2733] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:38:59 2022] DELETE /identity/v3/projects/c7b3761b958a442a96a8e593e783b4bd => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:38:59.399251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7df6f-813c-4e8f-a28b-a9f318932aaa None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:59.399747 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7df6f-813c-4e8f-a28b-a9f318932aaa None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:59.399864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7df6f-813c-4e8f-a28b-a9f318932aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:59.419473 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-7aa7df6f-813c-4e8f-a28b-a9f318932aaa None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:59.420685 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-7aa7df6f-813c-4e8f-a28b-a9f318932aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:59.481517 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1362/2734] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:59.491005 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029daa65-ea8e-484e-ae14-e0c08d0740ad None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:59.491296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029daa65-ea8e-484e-ae14-e0c08d0740ad None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:59.491599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029daa65-ea8e-484e-ae14-e0c08d0740ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:59.511893 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-029daa65-ea8e-484e-ae14-e0c08d0740ad None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:38:59.513107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-029daa65-ea8e-484e-ae14-e0c08d0740ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:59.521960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1373/2735] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:38:59.526277 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7df6f-813c-4e8f-a28b-a9f318932aaa None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:59.536115 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7df6f-813c-4e8f-a28b-a9f318932aaa None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:59.540732 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccdf84-e6a2-4844-96aa-d59ea0325999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:59.541316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccdf84-e6a2-4844-96aa-d59ea0325999 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:59.541629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccdf84-e6a2-4844-96aa-d59ea0325999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:59.541905 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccdf84-e6a2-4844-96aa-d59ea0325999 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:59.542551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccdf84-e6a2-4844-96aa-d59ea0325999 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:38:59.542906 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccdf84-e6a2-4844-96aa-d59ea0325999 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:59.544539 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccdf84-e6a2-4844-96aa-d59ea0325999 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:59.556288 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1363/2736] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:38:59.560738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029daa65-ea8e-484e-ae14-e0c08d0740ad None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:59.571075 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029daa65-ea8e-484e-ae14-e0c08d0740ad None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:59.575615 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3df0c-d6b8-4c6a-991a-96601604ebf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:59.576211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3df0c-d6b8-4c6a-991a-96601604ebf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:59.576487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3df0c-d6b8-4c6a-991a-96601604ebf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:59.576754 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3df0c-d6b8-4c6a-991a-96601604ebf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:59.577375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3df0c-d6b8-4c6a-991a-96601604ebf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:59.580207 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3df0c-d6b8-4c6a-991a-96601604ebf2 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:59.667287 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1374/2737] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:38:59 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:38:59.671539 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccdf84-e6a2-4844-96aa-d59ea0325999 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:59.681497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccdf84-e6a2-4844-96aa-d59ea0325999 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:59.739138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04939a0-9f09-4266-b4d6-f8e7260ce33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:59.739659 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04939a0-9f09-4266-b4d6-f8e7260ce33c admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:59.739936 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04939a0-9f09-4266-b4d6-f8e7260ce33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:59.740208 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04939a0-9f09-4266-b4d6-f8e7260ce33c admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:59.740838 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04939a0-9f09-4266-b4d6-f8e7260ce33c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:59.742755 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04939a0-9f09-4266-b4d6-f8e7260ce33c admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:59.828415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1364/2738] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:38:59 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:38:59.829933 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3df0c-d6b8-4c6a-991a-96601604ebf2 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:59.840510 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3df0c-d6b8-4c6a-991a-96601604ebf2 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:59.845486 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e91144-d413-4c8a-be6d-c4d7f6c79dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:59.846126 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e91144-d413-4c8a-be6d-c4d7f6c79dce admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:59.846409 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e91144-d413-4c8a-be6d-c4d7f6c79dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:59.846685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e91144-d413-4c8a-be6d-c4d7f6c79dce admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:59.847369 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e91144-d413-4c8a-be6d-c4d7f6c79dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:59.848961 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e91144-d413-4c8a-be6d-c4d7f6c79dce admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:59.862726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1375/2739] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:59.864164 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04939a0-9f09-4266-b4d6-f8e7260ce33c admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:59.874173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04939a0-9f09-4266-b4d6-f8e7260ce33c admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:59.879803 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0249218-4709-480b-b01f-52c855cea11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:59.880399 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0249218-4709-480b-b01f-52c855cea11a admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:59.880679 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0249218-4709-480b-b01f-52c855cea11a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:59.880951 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0249218-4709-480b-b01f-52c855cea11a admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:59.881587 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0249218-4709-480b-b01f-52c855cea11a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:59.883203 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0249218-4709-480b-b01f-52c855cea11a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:59.894280 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1365/2740] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:38:59.898417 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e91144-d413-4c8a-be6d-c4d7f6c79dce admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:59.904847 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:59.908686 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e91144-d413-4c8a-be6d-c4d7f6c79dce admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:59.913280 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4e301-5858-4d3f-8f5e-cc83e73a41db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:59.919078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4e301-5858-4d3f-8f5e-cc83e73a41db admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:59.919078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4e301-5858-4d3f-8f5e-cc83e73a41db admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:59.919078 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4e301-5858-4d3f-8f5e-cc83e73a41db admin admin] PATH_INFO: `/v3/projects/01646c09ff1a45b5a3919ec2fd0a148f/users/2fc4f779db74418488f41e1444325f8a/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:59.929681 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4e301-5858-4d3f-8f5e-cc83e73a41db admin admin] RBAC: Authorizing `identity:create_grant(project_id=01646c09ff1a45b5a3919ec2fd0a148f, user_id=2fc4f779db74418488f41e1444325f8a, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:59.932439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4e301-5858-4d3f-8f5e-cc83e73a41db admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:00.019405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1376/2741] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:38:59 2022] PUT /identity/v3/projects/01646c09ff1a45b5a3919ec2fd0a148f/users/2fc4f779db74418488f41e1444325f8a/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:39:00.029738 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcbd8d-cc1f-42df-a73f-e158cff1db72 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:00.030027 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcbd8d-cc1f-42df-a73f-e158cff1db72 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:00.030299 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcbd8d-cc1f-42df-a73f-e158cff1db72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:00.059443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-0ffcbd8d-cc1f-42df-a73f-e158cff1db72 None None] MFA Rules not processed for user `2fc4f779db74418488f41e1444325f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:39:00.060650 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-0ffcbd8d-cc1f-42df-a73f-e158cff1db72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:00.079136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:39:00.130827 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1366/2742] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:39:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:39:00.138211 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4e301-5858-4d3f-8f5e-cc83e73a41db admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:00.139950 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-64d4e301-5858-4d3f-8f5e-cc83e73a41db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:00.197425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4e301-5858-4d3f-8f5e-cc83e73a41db admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:00.201263 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1dab38-60e9-443a-aea6-afe554e29380 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:00.201875 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1dab38-60e9-443a-aea6-afe554e29380 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:00.202157 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1dab38-60e9-443a-aea6-afe554e29380 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:00.202428 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1dab38-60e9-443a-aea6-afe554e29380 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:00.211148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1dab38-60e9-443a-aea6-afe554e29380 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:00.212573 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1dab38-60e9-443a-aea6-afe554e29380 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:00.221855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1377/2743] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:39:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:39:10.173968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=19 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:39:10.217275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=34,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:39:10.515425 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcbd8d-cc1f-42df-a73f-e158cff1db72 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:10.580523 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcbd8d-cc1f-42df-a73f-e158cff1db72 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:10.586082 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5a646-596e-402c-a8ae-9f4839833e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:10.589373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5a646-596e-402c-a8ae-9f4839833e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:10.589373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5a646-596e-402c-a8ae-9f4839833e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:10.589373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5a646-596e-402c-a8ae-9f4839833e34 admin admin] PATH_INFO: `/v3/users/2fc4f779db74418488f41e1444325f8a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:10.589373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5a646-596e-402c-a8ae-9f4839833e34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc4f779db74418488f41e1444325f8a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:10.590393 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5a646-596e-402c-a8ae-9f4839833e34 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:10.741675 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4ac5a646-596e-402c-a8ae-9f4839833e34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc4f779db74418488f41e1444325f8a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:10.802110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4ac5a646-596e-402c-a8ae-9f4839833e34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc4f779db74418488f41e1444325f8a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:10.802744 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4ac5a646-596e-402c-a8ae-9f4839833e34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc4f779db74418488f41e1444325f8a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:10.815468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4ac5a646-596e-402c-a8ae-9f4839833e34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc4f779db74418488f41e1444325f8a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:10.816142 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4ac5a646-596e-402c-a8ae-9f4839833e34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc4f779db74418488f41e1444325f8a'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:10.829055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1367/2744] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:39:10 2022] DELETE /identity/v3/users/2fc4f779db74418488f41e1444325f8a => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:39:10.834634 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb4a98-9b72-48ea-bd83-9430b418337d None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:10.834946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb4a98-9b72-48ea-bd83-9430b418337d None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:10.835225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb4a98-9b72-48ea-bd83-9430b418337d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:10.854701 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-9bfb4a98-9b72-48ea-bd83-9430b418337d None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:39:10.855954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9bfb4a98-9b72-48ea-bd83-9430b418337d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:10.914155 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1378/2745] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:39:10.924269 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5a646-596e-402c-a8ae-9f4839833e34 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:10.925880 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4ac5a646-596e-402c-a8ae-9f4839833e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:10.991381 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5a646-596e-402c-a8ae-9f4839833e34 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:10.995227 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d948c7-afb1-4015-9e22-a7ef99c449be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:10.995768 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d948c7-afb1-4015-9e22-a7ef99c449be service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:10.996046 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d948c7-afb1-4015-9e22-a7ef99c449be service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:10.996316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d948c7-afb1-4015-9e22-a7ef99c449be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:11.005862 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d948c7-afb1-4015-9e22-a7ef99c449be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:11.007320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d948c7-afb1-4015-9e22-a7ef99c449be service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:11.016464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1368/2746] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:39:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:39:11.426349 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb4a98-9b72-48ea-bd83-9430b418337d None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:11.431230 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9bfb4a98-9b72-48ea-bd83-9430b418337d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:11.441061 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb4a98-9b72-48ea-bd83-9430b418337d None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:11.445961 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2f9af-ce2b-4918-97ed-e0c842d5958e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:11.446636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2f9af-ce2b-4918-97ed-e0c842d5958e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:11.446939 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2f9af-ce2b-4918-97ed-e0c842d5958e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:11.447219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2f9af-ce2b-4918-97ed-e0c842d5958e admin admin] PATH_INFO: `/v3/projects/01646c09ff1a45b5a3919ec2fd0a148f` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:11.448490 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2f9af-ce2b-4918-97ed-e0c842d5958e admin admin] RBAC: Authorizing `identity:delete_project(project_id=01646c09ff1a45b5a3919ec2fd0a148f)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:11.450403 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2f9af-ce2b-4918-97ed-e0c842d5958e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:11.604059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-fba2f9af-ce2b-4918-97ed-e0c842d5958e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01646c09ff1a45b5a3919ec2fd0a148f', 'request_id': 'req-fba2f9af-ce2b-4918-97ed-e0c842d5958e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:11.609448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-fba2f9af-ce2b-4918-97ed-e0c842d5958e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01646c09ff1a45b5a3919ec2fd0a148f', 'request_id': 'req-fba2f9af-ce2b-4918-97ed-e0c842d5958e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:11.687623 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-fba2f9af-ce2b-4918-97ed-e0c842d5958e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01646c09ff1a45b5a3919ec2fd0a148f', 'request_id': 'req-fba2f9af-ce2b-4918-97ed-e0c842d5958e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:11.699776 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-fba2f9af-ce2b-4918-97ed-e0c842d5958e admin admin] The token cache is being invalidate because project 01646c09ff1a45b5a3919ec2fd0a148f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:39:11.700070 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-fba2f9af-ce2b-4918-97ed-e0c842d5958e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:11.700708 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-fba2f9af-ce2b-4918-97ed-e0c842d5958e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:11.702228 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1379/2747] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:39:11 2022] DELETE /identity/v3/projects/01646c09ff1a45b5a3919ec2fd0a148f => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:39:21.012438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:39:21.697161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:39:27.882686 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d948c7-afb1-4015-9e22-a7ef99c449be service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:27.884247 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-f5d948c7-afb1-4015-9e22-a7ef99c449be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:27.897591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d948c7-afb1-4015-9e22-a7ef99c449be service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:27.955413 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f526b8e-8fb6-40b8-bc34-82668ef016ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:27.956062 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f526b8e-8fb6-40b8-bc34-82668ef016ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:27.956350 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f526b8e-8fb6-40b8-bc34-82668ef016ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:27.956626 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f526b8e-8fb6-40b8-bc34-82668ef016ac admin admin] PATH_INFO: `/v3/users/f5dfab772b684183bcc244495f6b9f16` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:27.957765 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f526b8e-8fb6-40b8-bc34-82668ef016ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5dfab772b684183bcc244495f6b9f16)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:27.959703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f526b8e-8fb6-40b8-bc34-82668ef016ac admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:28.130992 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4f526b8e-8fb6-40b8-bc34-82668ef016ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5dfab772b684183bcc244495f6b9f16'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:28.184754 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4f526b8e-8fb6-40b8-bc34-82668ef016ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5dfab772b684183bcc244495f6b9f16'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:28.185412 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4f526b8e-8fb6-40b8-bc34-82668ef016ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5dfab772b684183bcc244495f6b9f16'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:28.198427 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4f526b8e-8fb6-40b8-bc34-82668ef016ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5dfab772b684183bcc244495f6b9f16'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:28.199343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-4f526b8e-8fb6-40b8-bc34-82668ef016ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5dfab772b684183bcc244495f6b9f16'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:28.212300 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1369/2748] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 19:39:27 2022] DELETE /identity/v3/users/f5dfab772b684183bcc244495f6b9f16 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:39:28.217258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7224a-1fd2-4621-bde7-72088adabc49 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:28.218186 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7224a-1fd2-4621-bde7-72088adabc49 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:28.218667 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7224a-1fd2-4621-bde7-72088adabc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:28.238425 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-ffd7224a-1fd2-4621-bde7-72088adabc49 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:39:28.239895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ffd7224a-1fd2-4621-bde7-72088adabc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:28.299558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1380/2749] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:39:28.314827 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f526b8e-8fb6-40b8-bc34-82668ef016ac admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:28.316231 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-4f526b8e-8fb6-40b8-bc34-82668ef016ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:28.375420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f526b8e-8fb6-40b8-bc34-82668ef016ac admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:28.379077 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5347a-e49f-44e7-a044-6afff6a7f777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:28.379683 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5347a-e49f-44e7-a044-6afff6a7f777 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:28.379955 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5347a-e49f-44e7-a044-6afff6a7f777 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:28.380222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5347a-e49f-44e7-a044-6afff6a7f777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:28.390926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5347a-e49f-44e7-a044-6afff6a7f777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:28.392318 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5347a-e49f-44e7-a044-6afff6a7f777 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:28.403835 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1370/2750] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:39:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:39:28.757983 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7224a-1fd2-4621-bde7-72088adabc49 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:28.759552 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ffd7224a-1fd2-4621-bde7-72088adabc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:28.772498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7224a-1fd2-4621-bde7-72088adabc49 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:28.777358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9c216-e4df-41c8-9bb0-068858ed894f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:28.777993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9c216-e4df-41c8-9bb0-068858ed894f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:28.778279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9c216-e4df-41c8-9bb0-068858ed894f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:28.778557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9c216-e4df-41c8-9bb0-068858ed894f admin admin] PATH_INFO: `/v3/projects/a091bdeaca5f4d4e816ce658b3df6ed4` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:28.779735 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9c216-e4df-41c8-9bb0-068858ed894f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a091bdeaca5f4d4e816ce658b3df6ed4)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:28.781731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9c216-e4df-41c8-9bb0-068858ed894f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:28.877095 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-19f9c216-e4df-41c8-9bb0-068858ed894f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a091bdeaca5f4d4e816ce658b3df6ed4', 'request_id': 'req-19f9c216-e4df-41c8-9bb0-068858ed894f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:28.882536 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-19f9c216-e4df-41c8-9bb0-068858ed894f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a091bdeaca5f4d4e816ce658b3df6ed4', 'request_id': 'req-19f9c216-e4df-41c8-9bb0-068858ed894f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:28.927325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-19f9c216-e4df-41c8-9bb0-068858ed894f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a091bdeaca5f4d4e816ce658b3df6ed4', 'request_id': 'req-19f9c216-e4df-41c8-9bb0-068858ed894f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:28.939479 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-19f9c216-e4df-41c8-9bb0-068858ed894f admin admin] The token cache is being invalidate because project a091bdeaca5f4d4e816ce658b3df6ed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:39:28.940725 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-19f9c216-e4df-41c8-9bb0-068858ed894f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:28.940725 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-19f9c216-e4df-41c8-9bb0-068858ed894f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:39:28.941840 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1381/2751] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 19:39:28 2022] DELETE /identity/v3/projects/a091bdeaca5f4d4e816ce658b3df6ed4 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:39:28.947378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71524be-4854-4add-9e36-faa8ac4545e7 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:28.947898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71524be-4854-4add-9e36-faa8ac4545e7 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:28.947898 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71524be-4854-4add-9e36-faa8ac4545e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:28.968718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-e71524be-4854-4add-9e36-faa8ac4545e7 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:39:28.969913 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e71524be-4854-4add-9e36-faa8ac4545e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:28.986285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:39:29.029629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1371/2752] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:39:29.037946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ebce3-4fca-4d15-9a00-3690e4fef89a None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:29.038326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ebce3-4fca-4d15-9a00-3690e4fef89a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:29.038564 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ebce3-4fca-4d15-9a00-3690e4fef89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:29.060436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-9b2ebce3-4fca-4d15-9a00-3690e4fef89a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:39:29.061651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9b2ebce3-4fca-4d15-9a00-3690e4fef89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:29.069753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1382/2753] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:39:29.074538 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71524be-4854-4add-9e36-faa8ac4545e7 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:29.084864 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71524be-4854-4add-9e36-faa8ac4545e7 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:29.090425 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea77fc-1920-4eab-baa3-32c5564966fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:29.091079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea77fc-1920-4eab-baa3-32c5564966fb admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:29.091362 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea77fc-1920-4eab-baa3-32c5564966fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:29.091632 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea77fc-1920-4eab-baa3-32c5564966fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:29.092276 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea77fc-1920-4eab-baa3-32c5564966fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:39:29.092582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea77fc-1920-4eab-baa3-32c5564966fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:29.094158 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea77fc-1920-4eab-baa3-32c5564966fb admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:29.105806 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1372/2754] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 19:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:39:29.110084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ebce3-4fca-4d15-9a00-3690e4fef89a None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:29.119291 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ebce3-4fca-4d15-9a00-3690e4fef89a None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:29.123905 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a546444-973c-4068-a23e-92d8288ed378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:29.124498 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a546444-973c-4068-a23e-92d8288ed378 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:29.124777 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a546444-973c-4068-a23e-92d8288ed378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:29.125051 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a546444-973c-4068-a23e-92d8288ed378 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:29.125695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a546444-973c-4068-a23e-92d8288ed378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:29.127617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a546444-973c-4068-a23e-92d8288ed378 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:29.197658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1383/2755] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 19:39:29 2022] POST /identity/v3/projects => generated 381 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:39:29.201764 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea77fc-1920-4eab-baa3-32c5564966fb admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:29.212437 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea77fc-1920-4eab-baa3-32c5564966fb admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:29.271236 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af86c6-9c51-4e31-865e-c1c56365141d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:29.271852 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af86c6-9c51-4e31-865e-c1c56365141d admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:29.272136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af86c6-9c51-4e31-865e-c1c56365141d admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:29.272412 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af86c6-9c51-4e31-865e-c1c56365141d admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:29.273061 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af86c6-9c51-4e31-865e-c1c56365141d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:29.275128 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af86c6-9c51-4e31-865e-c1c56365141d admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:29.379983 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1373/2756] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 19:39:29 2022] POST /identity/v3/users => generated 291 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:39:29.384346 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a546444-973c-4068-a23e-92d8288ed378 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:29.395677 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a546444-973c-4068-a23e-92d8288ed378 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:29.400454 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bbd33-e8ee-489e-ae1f-2e1d57f34d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:29.401073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bbd33-e8ee-489e-ae1f-2e1d57f34d07 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:29.401357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bbd33-e8ee-489e-ae1f-2e1d57f34d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:29.401634 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bbd33-e8ee-489e-ae1f-2e1d57f34d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:29.403044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79bbd33-e8ee-489e-ae1f-2e1d57f34d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:29.404723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79bbd33-e8ee-489e-ae1f-2e1d57f34d07 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:29.415202 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1384/2757] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:39:29.419477 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af86c6-9c51-4e31-865e-c1c56365141d admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:29.429610 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af86c6-9c51-4e31-865e-c1c56365141d admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:29.434904 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6add909-eb56-4697-8d86-16f32572bd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:29.435513 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6add909-eb56-4697-8d86-16f32572bd78 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:29.435813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6add909-eb56-4697-8d86-16f32572bd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:29.436078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6add909-eb56-4697-8d86-16f32572bd78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:29.436721 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6add909-eb56-4697-8d86-16f32572bd78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:29.438317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6add909-eb56-4697-8d86-16f32572bd78 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:29.450004 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1374/2758] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 19:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:39:29.454307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bbd33-e8ee-489e-ae1f-2e1d57f34d07 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:29.464209 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bbd33-e8ee-489e-ae1f-2e1d57f34d07 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:29.469023 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbf1da-03ef-4058-ae55-461e08602e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:29.469679 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbf1da-03ef-4058-ae55-461e08602e7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:29.469964 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbf1da-03ef-4058-ae55-461e08602e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:29.470240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbf1da-03ef-4058-ae55-461e08602e7b admin admin] PATH_INFO: `/v3/projects/3374fda464574c07a137d5fe03b02712/users/c8214c2eb78f404d94bfc73ca7139e78/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:29.485152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cbf1da-03ef-4058-ae55-461e08602e7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3374fda464574c07a137d5fe03b02712, user_id=c8214c2eb78f404d94bfc73ca7139e78, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:29.487849 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cbf1da-03ef-4058-ae55-461e08602e7b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:29.555644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1385/2759] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 19:39:29 2022] PUT /identity/v3/projects/3374fda464574c07a137d5fe03b02712/users/c8214c2eb78f404d94bfc73ca7139e78/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:39:29.561798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69f4fd-110c-40f1-8ec3-77e66500cf57 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:29.562089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69f4fd-110c-40f1-8ec3-77e66500cf57 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:29.562358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69f4fd-110c-40f1-8ec3-77e66500cf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:29.582581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-2f69f4fd-110c-40f1-8ec3-77e66500cf57 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:39:29.583819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-2f69f4fd-110c-40f1-8ec3-77e66500cf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:29.643891 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1375/2760] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:39:29.653972 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbf1da-03ef-4058-ae55-461e08602e7b admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:29.655766 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-28cbf1da-03ef-4058-ae55-461e08602e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:29.677634 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:39:29.714344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbf1da-03ef-4058-ae55-461e08602e7b admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:29.718170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cedd0c-9d01-4dca-99ec-019a20d6fa13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:29.721800 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cedd0c-9d01-4dca-99ec-019a20d6fa13 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:29.721800 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cedd0c-9d01-4dca-99ec-019a20d6fa13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:29.721800 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cedd0c-9d01-4dca-99ec-019a20d6fa13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:29.728493 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cedd0c-9d01-4dca-99ec-019a20d6fa13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:29.729921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cedd0c-9d01-4dca-99ec-019a20d6fa13 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:29.739673 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1386/2761] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:39:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:39:31.128817 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4291f9c-b0a6-46e9-acb4-407d2b017733 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:31.129398 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4291f9c-b0a6-46e9-acb4-407d2b017733 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:31.129498 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4291f9c-b0a6-46e9-acb4-407d2b017733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:31.165842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-a4291f9c-b0a6-46e9-acb4-407d2b017733 None None] MFA Rules not processed for user `c8214c2eb78f404d94bfc73ca7139e78`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:39:31.167150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a4291f9c-b0a6-46e9-acb4-407d2b017733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:31.185215 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:39:31.238161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1376/2762] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:39:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:39:31.251518 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cedd0c-9d01-4dca-99ec-019a20d6fa13 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:31.251965 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a7cedd0c-9d01-4dca-99ec-019a20d6fa13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:31.313190 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cedd0c-9d01-4dca-99ec-019a20d6fa13 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:31.316846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924fbaa-4043-4397-ad8e-98e1573126f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:31.317447 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924fbaa-4043-4397-ad8e-98e1573126f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:31.317725 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924fbaa-4043-4397-ad8e-98e1573126f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:31.318047 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924fbaa-4043-4397-ad8e-98e1573126f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:31.327343 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924fbaa-4043-4397-ad8e-98e1573126f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:31.329055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924fbaa-4043-4397-ad8e-98e1573126f0 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:31.338507 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1387/2763] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:39:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:39:34.095812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cf031-ad3d-4dc3-859d-5925e49be25e None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:34.096593 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cf031-ad3d-4dc3-859d-5925e49be25e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:34.096593 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cf031-ad3d-4dc3-859d-5925e49be25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:34.116496 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-e06cf031-ad3d-4dc3-859d-5925e49be25e None None] MFA Rules not processed for user `c8214c2eb78f404d94bfc73ca7139e78`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:39:34.117726 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e06cf031-ad3d-4dc3-859d-5925e49be25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:34.125641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1377/2764] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:39:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:39:41.334105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=43 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:39:44.110987 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=22 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:40:12.734946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924fbaa-4043-4397-ad8e-98e1573126f0 service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:40:12.736399 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f924fbaa-4043-4397-ad8e-98e1573126f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:40:12.749405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924fbaa-4043-4397-ad8e-98e1573126f0 service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:40:12.753090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83eb638-4b40-45e3-ae76-24eea622ac3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:40:12.754312 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83eb638-4b40-45e3-ae76-24eea622ac3c service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:40:12.754658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83eb638-4b40-45e3-ae76-24eea622ac3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:40:12.754962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83eb638-4b40-45e3-ae76-24eea622ac3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:40:12.756703 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c83eb638-4b40-45e3-ae76-24eea622ac3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:40:12.828297 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83eb638-4b40-45e3-ae76-24eea622ac3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:40:12.829767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83eb638-4b40-45e3-ae76-24eea622ac3c service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:40:12.841811 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1388/2765] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:40:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:40:22.836691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=34 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:41:21.472413 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cf031-ad3d-4dc3-859d-5925e49be25e None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:41:21.473867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e06cf031-ad3d-4dc3-859d-5925e49be25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:41:21.486419 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cf031-ad3d-4dc3-859d-5925e49be25e None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:41:21.490021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d396df4-e2ba-4790-9ac4-5c17390f6400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:41:21.490624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d396df4-e2ba-4790-9ac4-5c17390f6400 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:41:21.490968 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d396df4-e2ba-4790-9ac4-5c17390f6400 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:41:21.491251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d396df4-e2ba-4790-9ac4-5c17390f6400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:41:21.492999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-5d396df4-e2ba-4790-9ac4-5c17390f6400 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:41:21.503257 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d396df4-e2ba-4790-9ac4-5c17390f6400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:41:21.504719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d396df4-e2ba-4790-9ac4-5c17390f6400 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:41:21.513964 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1378/2766] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:41:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:41:22.548661 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83eb638-4b40-45e3-ae76-24eea622ac3c service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:41:22.558519 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83eb638-4b40-45e3-ae76-24eea622ac3c service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:41:22.563189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd3716-4dbb-45b5-bd0f-972f31a455b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:41:22.563842 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd3716-4dbb-45b5-bd0f-972f31a455b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:41:22.564130 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd3716-4dbb-45b5-bd0f-972f31a455b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:41:22.564579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd3716-4dbb-45b5-bd0f-972f31a455b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:41:22.566472 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-29dd3716-4dbb-45b5-bd0f-972f31a455b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:41:22.581883 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd3716-4dbb-45b5-bd0f-972f31a455b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:41:22.583450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd3716-4dbb-45b5-bd0f-972f31a455b3 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:41:22.596497 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1389/2767] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:41:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:41:31.510227 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:41:32.590291 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=6 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:43:04.337474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d396df4-e2ba-4790-9ac4-5c17390f6400 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:43:04.339225 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-5d396df4-e2ba-4790-9ac4-5c17390f6400 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:43:04.434036 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d396df4-e2ba-4790-9ac4-5c17390f6400 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:43:04.440340 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d26fa-3d14-4845-937e-24066004dbef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:43:04.440946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d26fa-3d14-4845-937e-24066004dbef service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:43:04.441243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d26fa-3d14-4845-937e-24066004dbef service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:43:04.441500 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d26fa-3d14-4845-937e-24066004dbef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:43:04.443306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-6e0d26fa-3d14-4845-937e-24066004dbef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:43:04.522387 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d26fa-3d14-4845-937e-24066004dbef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:43:04.524025 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d26fa-3d14-4845-937e-24066004dbef service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:43:04.543430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1379/2768] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:43:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:43:14.540648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=75 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:43:17.708076 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd3716-4dbb-45b5-bd0f-972f31a455b3 service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:43:17.709481 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-29dd3716-4dbb-45b5-bd0f-972f31a455b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:43:17.849675 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd3716-4dbb-45b5-bd0f-972f31a455b3 service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:43:17.854835 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9d862-2388-465d-87f3-76a2f4ee529c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:43:17.855602 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9d862-2388-465d-87f3-76a2f4ee529c service placement] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:43:17.855744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9d862-2388-465d-87f3-76a2f4ee529c service placement] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:43:17.856010 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9d862-2388-465d-87f3-76a2f4ee529c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:43:17.857742 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-68a9d862-2388-465d-87f3-76a2f4ee529c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:43:17.869585 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9d862-2388-465d-87f3-76a2f4ee529c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:43:17.871196 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9d862-2388-465d-87f3-76a2f4ee529c service placement] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:43:17.888543 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1390/2769] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:43:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:43:27.876935 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=63 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:44:31.583855 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d26fa-3d14-4845-937e-24066004dbef service nova] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:44:31.603829 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d26fa-3d14-4845-937e-24066004dbef service nova] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:44:31.607468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdc9ca-7a42-4639-993d-24918fc16c5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:44:31.608067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc9ca-7a42-4639-993d-24918fc16c5b service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:44:31.608346 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc9ca-7a42-4639-993d-24918fc16c5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:44:31.608620 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc9ca-7a42-4639-993d-24918fc16c5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:44:31.617138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdc9ca-7a42-4639-993d-24918fc16c5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:44:31.618555 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdc9ca-7a42-4639-993d-24918fc16c5b service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:44:31.627934 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1380/2770] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:44:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:44:41.624061 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=10 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:46:22.808255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9d862-2388-465d-87f3-76a2f4ee529c service placement] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:46:22.817652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9d862-2388-465d-87f3-76a2f4ee529c service placement] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:46:22.821512 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed180837-c934-43d7-95df-a0c063e9677f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:46:22.822897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed180837-c934-43d7-95df-a0c063e9677f service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:46:22.823202 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed180837-c934-43d7-95df-a0c063e9677f service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:46:22.823475 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed180837-c934-43d7-95df-a0c063e9677f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:46:22.833800 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed180837-c934-43d7-95df-a0c063e9677f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:46:22.835251 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed180837-c934-43d7-95df-a0c063e9677f service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:46:22.846036 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1391/2771] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:46:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:46:32.841316 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=6 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:47:17.090509 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdc9ca-7a42-4639-993d-24918fc16c5b service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:47:17.112390 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdc9ca-7a42-4639-993d-24918fc16c5b service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:47:17.116170 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8ad20-842a-4cb8-9604-6dc52dd627ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:47:17.117272 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8ad20-842a-4cb8-9604-6dc52dd627ba service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:47:17.117611 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8ad20-842a-4cb8-9604-6dc52dd627ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:47:17.118270 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8ad20-842a-4cb8-9604-6dc52dd627ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:47:17.126220 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8ad20-842a-4cb8-9604-6dc52dd627ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:47:17.127703 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8ad20-842a-4cb8-9604-6dc52dd627ba service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:47:17.135885 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1381/2772] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:47:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:47:27.132113 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=10 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:48:04.616459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed180837-c934-43d7-95df-a0c063e9677f service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:48:04.625131 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed180837-c934-43d7-95df-a0c063e9677f service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:48:04.628770 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300572bd-e783-4c99-bf95-5495f0a915f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:48:04.629362 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300572bd-e783-4c99-bf95-5495f0a915f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:48:04.629642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300572bd-e783-4c99-bf95-5495f0a915f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:48:04.629916 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300572bd-e783-4c99-bf95-5495f0a915f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:48:04.639057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300572bd-e783-4c99-bf95-5495f0a915f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:48:04.640470 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300572bd-e783-4c99-bf95-5495f0a915f0 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:48:04.649392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1392/2773] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:48:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:48:14.645229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=6 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:48:21.411641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8ad20-842a-4cb8-9604-6dc52dd627ba service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:48:21.424608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8ad20-842a-4cb8-9604-6dc52dd627ba service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:48:21.430425 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9ba67-1e78-4c2a-b3da-a1ac62b8e012 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:48:21.431109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9ba67-1e78-4c2a-b3da-a1ac62b8e012 service placement] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:48:21.431389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9ba67-1e78-4c2a-b3da-a1ac62b8e012 service placement] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:48:21.431655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9ba67-1e78-4c2a-b3da-a1ac62b8e012 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:48:21.440505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9ba67-1e78-4c2a-b3da-a1ac62b8e012 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:48:21.442043 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9ba67-1e78-4c2a-b3da-a1ac62b8e012 service placement] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:48:21.452265 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1382/2774] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:48:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:48:31.447954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:49:31.737201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300572bd-e783-4c99-bf95-5495f0a915f0 service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:49:31.739392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-300572bd-e783-4c99-bf95-5495f0a915f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:49:31.806614 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300572bd-e783-4c99-bf95-5495f0a915f0 service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:49:31.810202 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac33bf-8ce8-4e13-9e71-28e16743b215 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:49:31.810802 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac33bf-8ce8-4e13-9e71-28e16743b215 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:49:31.811111 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac33bf-8ce8-4e13-9e71-28e16743b215 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:49:31.811386 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac33bf-8ce8-4e13-9e71-28e16743b215 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:49:31.813195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-43ac33bf-8ce8-4e13-9e71-28e16743b215 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:49:31.893180 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac33bf-8ce8-4e13-9e71-28e16743b215 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:49:31.894528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac33bf-8ce8-4e13-9e71-28e16743b215 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:49:31.912843 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1393/2775] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:49:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:49:41.909843 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=66 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:52:23.929885 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9ba67-1e78-4c2a-b3da-a1ac62b8e012 service placement] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:52:23.931600 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-03a9ba67-1e78-4c2a-b3da-a1ac62b8e012 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:52:24.003231 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9ba67-1e78-4c2a-b3da-a1ac62b8e012 service placement] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:52:24.006775 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0079216-852e-436f-9607-46b665a51283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:52:24.007403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0079216-852e-436f-9607-46b665a51283 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:52:24.007676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0079216-852e-436f-9607-46b665a51283 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:52:24.007944 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0079216-852e-436f-9607-46b665a51283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:52:24.009750 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e0079216-852e-436f-9607-46b665a51283 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:52:24.074907 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac33bf-8ce8-4e13-9e71-28e16743b215 service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:52:24.082687 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0079216-852e-436f-9607-46b665a51283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:52:24.084164 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0079216-852e-436f-9607-46b665a51283 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:52:24.084300 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac33bf-8ce8-4e13-9e71-28e16743b215 service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:52:24.087796 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab66017-f7ef-4ab7-9107-57f8ec8172e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:52:24.088396 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab66017-f7ef-4ab7-9107-57f8ec8172e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:52:24.088675 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab66017-f7ef-4ab7-9107-57f8ec8172e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:52:24.088949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab66017-f7ef-4ab7-9107-57f8ec8172e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:52:24.091598 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4ab66017-f7ef-4ab7-9107-57f8ec8172e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:52:24.096163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1383/2776] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:52:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:52:24.102307 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab66017-f7ef-4ab7-9107-57f8ec8172e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:52:24.103815 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab66017-f7ef-4ab7-9107-57f8ec8172e7 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:52:24.114310 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1394/2777] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:52:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:52:25.325789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0079216-852e-436f-9607-46b665a51283 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:52:25.327527 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e0079216-852e-436f-9607-46b665a51283 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:52:25.338827 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0079216-852e-436f-9607-46b665a51283 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:52:25.343656 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b61a3-390c-4c37-a68b-2c253ed1473e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:52:25.344255 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b61a3-390c-4c37-a68b-2c253ed1473e service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:52:25.344530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b61a3-390c-4c37-a68b-2c253ed1473e service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:52:25.344800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b61a3-390c-4c37-a68b-2c253ed1473e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:52:25.346647 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a36b61a3-390c-4c37-a68b-2c253ed1473e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:52:25.361089 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b61a3-390c-4c37-a68b-2c253ed1473e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:52:25.362493 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b61a3-390c-4c37-a68b-2c253ed1473e service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:52:25.373585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1384/2778] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:52:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:52:34.110278 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=6 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:52:35.368510 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=65 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:53:05.391672 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab66017-f7ef-4ab7-9107-57f8ec8172e7 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:53:05.401021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab66017-f7ef-4ab7-9107-57f8ec8172e7 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:53:05.405365 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a46768-fa96-44cc-ae43-76379be9b789 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:53:05.405958 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a46768-fa96-44cc-ae43-76379be9b789 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:53:05.406236 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a46768-fa96-44cc-ae43-76379be9b789 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:53:05.406504 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a46768-fa96-44cc-ae43-76379be9b789 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:53:05.408484 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f1a46768-fa96-44cc-ae43-76379be9b789 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:53:05.488979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a46768-fa96-44cc-ae43-76379be9b789 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:53:05.490326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a46768-fa96-44cc-ae43-76379be9b789 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:53:05.509141 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1395/2779] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:53:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:53:15.506521 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=41 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:53:24.993245 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b61a3-390c-4c37-a68b-2c253ed1473e service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:53:24.994819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a36b61a3-390c-4c37-a68b-2c253ed1473e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:53:25.159264 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b61a3-390c-4c37-a68b-2c253ed1473e service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:53:25.164795 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ed6c9-a83b-4a94-b02c-08ffae218f54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:53:25.165500 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ed6c9-a83b-4a94-b02c-08ffae218f54 service placement] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:53:25.165846 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ed6c9-a83b-4a94-b02c-08ffae218f54 service placement] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:53:25.166200 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ed6c9-a83b-4a94-b02c-08ffae218f54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:53:25.168203 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-db4ed6c9-a83b-4a94-b02c-08ffae218f54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:53:25.181684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ed6c9-a83b-4a94-b02c-08ffae218f54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:53:25.183110 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ed6c9-a83b-4a94-b02c-08ffae218f54 service placement] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:53:25.195794 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1385/2780] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:53:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:53:26.115942 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a46768-fa96-44cc-ae43-76379be9b789 service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:53:26.117636 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f1a46768-fa96-44cc-ae43-76379be9b789 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:53:26.210574 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a46768-fa96-44cc-ae43-76379be9b789 service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:53:26.217374 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe92cae-23a3-45b1-8e3f-f35730a1c721 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:53:26.218173 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe92cae-23a3-45b1-8e3f-f35730a1c721 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:53:26.218455 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe92cae-23a3-45b1-8e3f-f35730a1c721 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:53:26.218738 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe92cae-23a3-45b1-8e3f-f35730a1c721 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:53:26.220604 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-fbe92cae-23a3-45b1-8e3f-f35730a1c721 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:53:26.234656 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe92cae-23a3-45b1-8e3f-f35730a1c721 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:53:26.236367 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe92cae-23a3-45b1-8e3f-f35730a1c721 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:53:26.251403 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1396/2781] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:53:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:53:35.191756 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=63 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:53:36.245043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=33 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:54:31.217432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ed6c9-a83b-4a94-b02c-08ffae218f54 service placement] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:54:31.230599 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ed6c9-a83b-4a94-b02c-08ffae218f54 service placement] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:54:31.236102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64249a61-f3eb-4d90-8771-1da8a6708ea1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:54:31.236698 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64249a61-f3eb-4d90-8771-1da8a6708ea1 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:54:31.236975 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64249a61-f3eb-4d90-8771-1da8a6708ea1 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:54:31.237246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64249a61-f3eb-4d90-8771-1da8a6708ea1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:54:31.246029 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64249a61-f3eb-4d90-8771-1da8a6708ea1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:54:31.247686 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64249a61-f3eb-4d90-8771-1da8a6708ea1 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:54:31.256799 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1386/2782] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:54:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:54:41.252633 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:57:24.069247 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe92cae-23a3-45b1-8e3f-f35730a1c721 service nova] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:57:24.090007 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe92cae-23a3-45b1-8e3f-f35730a1c721 service nova] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:57:24.093837 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867de6ae-98f7-46c3-85cb-07d83132c889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:57:24.094448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867de6ae-98f7-46c3-85cb-07d83132c889 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:57:24.094733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867de6ae-98f7-46c3-85cb-07d83132c889 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:57:24.095044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867de6ae-98f7-46c3-85cb-07d83132c889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:57:24.106748 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867de6ae-98f7-46c3-85cb-07d83132c889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:57:24.108214 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867de6ae-98f7-46c3-85cb-07d83132c889 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:57:24.120637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1397/2783] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:57:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:57:25.220123 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64249a61-f3eb-4d90-8771-1da8a6708ea1 service nova] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:57:25.230242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64249a61-f3eb-4d90-8771-1da8a6708ea1 service nova] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:57:25.234378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788b90a-a5a0-4c68-a461-ba6dcc997124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:57:25.235021 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788b90a-a5a0-4c68-a461-ba6dcc997124 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:57:25.235308 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788b90a-a5a0-4c68-a461-ba6dcc997124 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:57:25.235582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788b90a-a5a0-4c68-a461-ba6dcc997124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:57:25.246623 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788b90a-a5a0-4c68-a461-ba6dcc997124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:57:25.248076 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788b90a-a5a0-4c68-a461-ba6dcc997124 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:57:25.261011 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1387/2784] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:57:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:57:34.115298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=10 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:57:35.255044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:58:04.051795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867de6ae-98f7-46c3-85cb-07d83132c889 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:58:04.053202 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-867de6ae-98f7-46c3-85cb-07d83132c889 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:58:04.068914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867de6ae-98f7-46c3-85cb-07d83132c889 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:58:04.075098 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5b638-cbff-4ed9-ac13-cd162a2d5af3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:58:04.075707 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5b638-cbff-4ed9-ac13-cd162a2d5af3 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:58:04.075984 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5b638-cbff-4ed9-ac13-cd162a2d5af3 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:58:04.076340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5b638-cbff-4ed9-ac13-cd162a2d5af3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:58:04.084607 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5b638-cbff-4ed9-ac13-cd162a2d5af3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:58:04.088959 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5b638-cbff-4ed9-ac13-cd162a2d5af3 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:58:04.095534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1398/2785] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:58:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:58:14.091299 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=6 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:58:28.394073 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788b90a-a5a0-4c68-a461-ba6dcc997124 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:58:28.406007 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788b90a-a5a0-4c68-a461-ba6dcc997124 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:58:28.411291 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84df2a0-a721-410a-b169-d5660c3d8c05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:58:28.412553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84df2a0-a721-410a-b169-d5660c3d8c05 service placement] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:58:28.412553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84df2a0-a721-410a-b169-d5660c3d8c05 service placement] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:58:28.412553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84df2a0-a721-410a-b169-d5660c3d8c05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:58:28.422369 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84df2a0-a721-410a-b169-d5660c3d8c05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:58:28.424060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84df2a0-a721-410a-b169-d5660c3d8c05 service placement] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:58:28.435496 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1388/2786] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:58:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:58:38.431197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:59:27.411181 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7b8f2-07da-44c6-a838-c09e16722f80 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:59:27.411764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7b8f2-07da-44c6-a838-c09e16722f80 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:59:27.411876 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7b8f2-07da-44c6-a838-c09e16722f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:59:27.440452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-f0c7b8f2-07da-44c6-a838-c09e16722f80 None None] MFA Rules not processed for user `991f6d49ad014318bc9c2f5d57b9deb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:59:27.441719 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f0c7b8f2-07da-44c6-a838-c09e16722f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:59:27.449492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1399/2787] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 19:59:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:59:27.762029 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84df2a0-a721-410a-b169-d5660c3d8c05 service placement] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:59:27.771812 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84df2a0-a721-410a-b169-d5660c3d8c05 service placement] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:59:27.776086 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f464e4f-8fbd-497b-a201-24c36ea7db54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:59:27.776713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f464e4f-8fbd-497b-a201-24c36ea7db54 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:59:27.777044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f464e4f-8fbd-497b-a201-24c36ea7db54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:59:27.777265 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f464e4f-8fbd-497b-a201-24c36ea7db54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:59:27.779031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-0f464e4f-8fbd-497b-a201-24c36ea7db54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:59:27.788752 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f464e4f-8fbd-497b-a201-24c36ea7db54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:59:27.790164 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f464e4f-8fbd-497b-a201-24c36ea7db54 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:59:27.798491 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1389/2788] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:59:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:59:28.247513 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7b8f2-07da-44c6-a838-c09e16722f80 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:59:28.260339 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7b8f2-07da-44c6-a838-c09e16722f80 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:59:28.265292 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a5a59-f369-409f-a2c3-340926fd96f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:59:28.265907 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a5a59-f369-409f-a2c3-340926fd96f1 service placement] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:59:28.266189 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a5a59-f369-409f-a2c3-340926fd96f1 service placement] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:59:28.266464 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a5a59-f369-409f-a2c3-340926fd96f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:59:28.268235 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4c7a5a59-f369-409f-a2c3-340926fd96f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:59:28.281340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a5a59-f369-409f-a2c3-340926fd96f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:59:28.282891 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a5a59-f369-409f-a2c3-340926fd96f1 service placement] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:59:28.294546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1400/2789] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:59:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:59:32.010565 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f464e4f-8fbd-497b-a201-24c36ea7db54 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:59:32.019495 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f464e4f-8fbd-497b-a201-24c36ea7db54 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:59:32.023912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa4062-227e-4ce7-a0db-e293e8400949 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:59:32.024516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa4062-227e-4ce7-a0db-e293e8400949 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:59:32.024793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa4062-227e-4ce7-a0db-e293e8400949 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:59:32.025063 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa4062-227e-4ce7-a0db-e293e8400949 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:59:32.032981 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa4062-227e-4ce7-a0db-e293e8400949 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:59:32.034536 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa4062-227e-4ce7-a0db-e293e8400949 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:59:32.044173 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1390/2790] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:59:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:59:32.771853 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a5a59-f369-409f-a2c3-340926fd96f1 service placement] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:59:32.786332 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a5a59-f369-409f-a2c3-340926fd96f1 service placement] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:59:32.797333 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee12536-6ca8-4f67-8b5b-846a2e8447d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:59:32.798988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee12536-6ca8-4f67-8b5b-846a2e8447d2 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:59:32.799471 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee12536-6ca8-4f67-8b5b-846a2e8447d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:59:32.799753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee12536-6ca8-4f67-8b5b-846a2e8447d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:59:32.811311 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee12536-6ca8-4f67-8b5b-846a2e8447d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:59:32.813136 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee12536-6ca8-4f67-8b5b-846a2e8447d2 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:59:32.825960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1401/2791] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:59:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:59:39.093167 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa4062-227e-4ce7-a0db-e293e8400949 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:59:39.106197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa4062-227e-4ce7-a0db-e293e8400949 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:59:39.111681 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92110a7b-8a2d-4f9f-aa8a-8f6c4018d5c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:59:39.112272 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92110a7b-8a2d-4f9f-aa8a-8f6c4018d5c0 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:59:39.112551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92110a7b-8a2d-4f9f-aa8a-8f6c4018d5c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:59:39.112820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92110a7b-8a2d-4f9f-aa8a-8f6c4018d5c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:59:39.114593 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-92110a7b-8a2d-4f9f-aa8a-8f6c4018d5c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:59:39.126833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92110a7b-8a2d-4f9f-aa8a-8f6c4018d5c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:59:39.128408 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92110a7b-8a2d-4f9f-aa8a-8f6c4018d5c0 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:59:39.139361 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1391/2792] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:59:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:59:42.820258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=18 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:59:43.373426 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee12536-6ca8-4f67-8b5b-846a2e8447d2 service nova] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:59:43.385618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee12536-6ca8-4f67-8b5b-846a2e8447d2 service nova] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:59:43.391599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba725e1-acbe-43fc-89d5-628b9fc2c0e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:59:43.392239 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba725e1-acbe-43fc-89d5-628b9fc2c0e4 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:59:43.392525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba725e1-acbe-43fc-89d5-628b9fc2c0e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:59:43.392942 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba725e1-acbe-43fc-89d5-628b9fc2c0e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:59:43.394850 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-fba725e1-acbe-43fc-89d5-628b9fc2c0e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:59:43.408557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba725e1-acbe-43fc-89d5-628b9fc2c0e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:59:43.410163 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba725e1-acbe-43fc-89d5-628b9fc2c0e4 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:59:43.422047 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1402/2793] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 19:59:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:59:49.135046 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=18 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:59:53.417196 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=6 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:01:25.695987 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92110a7b-8a2d-4f9f-aa8a-8f6c4018d5c0 service nova] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:01:25.697597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-92110a7b-8a2d-4f9f-aa8a-8f6c4018d5c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:01:25.780316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92110a7b-8a2d-4f9f-aa8a-8f6c4018d5c0 service nova] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:01:25.783979 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abee323-f5f6-4bde-b982-4c357501aefb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:01:25.785247 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abee323-f5f6-4bde-b982-4c357501aefb service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:01:25.785516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abee323-f5f6-4bde-b982-4c357501aefb service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:01:25.785863 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abee323-f5f6-4bde-b982-4c357501aefb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:01:25.787771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-9abee323-f5f6-4bde-b982-4c357501aefb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:01:25.870195 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abee323-f5f6-4bde-b982-4c357501aefb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:01:25.871677 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abee323-f5f6-4bde-b982-4c357501aefb service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:01:25.890438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1392/2794] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:01:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:01:35.887888 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=70 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:03:26.864255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba725e1-acbe-43fc-89d5-628b9fc2c0e4 service nova] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:03:26.885996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba725e1-acbe-43fc-89d5-628b9fc2c0e4 service nova] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:03:26.889653 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5127fa-381b-4d81-93e9-e9241502f70e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:03:26.890263 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5127fa-381b-4d81-93e9-e9241502f70e service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:03:26.890543 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5127fa-381b-4d81-93e9-e9241502f70e service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:03:26.890840 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5127fa-381b-4d81-93e9-e9241502f70e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:03:26.892742 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-1d5127fa-381b-4d81-93e9-e9241502f70e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:03:27.001297 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5127fa-381b-4d81-93e9-e9241502f70e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:03:27.002757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5127fa-381b-4d81-93e9-e9241502f70e service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:03:27.021899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1403/2795] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:03:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:03:29.457467 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abee323-f5f6-4bde-b982-4c357501aefb service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:03:29.458933 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-9abee323-f5f6-4bde-b982-4c357501aefb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:03:29.601254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abee323-f5f6-4bde-b982-4c357501aefb service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:03:29.606712 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ce530-ecb5-4225-8b0f-a54438e36084 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:03:29.607350 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ce530-ecb5-4225-8b0f-a54438e36084 service placement] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:03:29.607630 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ce530-ecb5-4225-8b0f-a54438e36084 service placement] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:03:29.607903 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ce530-ecb5-4225-8b0f-a54438e36084 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:03:29.610341 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-230ce530-ecb5-4225-8b0f-a54438e36084 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:03:29.622330 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ce530-ecb5-4225-8b0f-a54438e36084 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:03:29.623937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ce530-ecb5-4225-8b0f-a54438e36084 service placement] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:03:29.634495 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1393/2796] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:03:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:03:37.019022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=56 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:03:39.629631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=63 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:04:37.064605 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5127fa-381b-4d81-93e9-e9241502f70e service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:04:37.066279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-1d5127fa-381b-4d81-93e9-e9241502f70e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:04:37.138951 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5127fa-381b-4d81-93e9-e9241502f70e service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:04:37.142762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba11568-c920-4812-ba16-787f131ac103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:04:37.143418 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba11568-c920-4812-ba16-787f131ac103 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:04:37.143695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba11568-c920-4812-ba16-787f131ac103 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:04:37.143996 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba11568-c920-4812-ba16-787f131ac103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:04:37.145861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-dba11568-c920-4812-ba16-787f131ac103 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:04:37.155482 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba11568-c920-4812-ba16-787f131ac103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:04:37.156892 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba11568-c920-4812-ba16-787f131ac103 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:04:37.165226 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1404/2797] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:04:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:04:47.161120 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=31 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:06:25.116965 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ce530-ecb5-4225-8b0f-a54438e36084 service placement] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:06:25.128576 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ce530-ecb5-4225-8b0f-a54438e36084 service placement] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:06:25.132320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fbe0a-c8b9-4243-9175-fd8f95f23165 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:06:25.132916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fbe0a-c8b9-4243-9175-fd8f95f23165 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:06:25.133199 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fbe0a-c8b9-4243-9175-fd8f95f23165 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:06:25.133474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fbe0a-c8b9-4243-9175-fd8f95f23165 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:06:25.141719 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fbe0a-c8b9-4243-9175-fd8f95f23165 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:06:25.143609 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fbe0a-c8b9-4243-9175-fd8f95f23165 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:06:25.152062 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1394/2798] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:06:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:06:35.148147 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:07:26.605843 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba11568-c920-4812-ba16-787f131ac103 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:07:26.625072 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba11568-c920-4812-ba16-787f131ac103 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:07:26.628660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d4de3-be08-4f17-99f4-8bdec6add915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:07:26.629251 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d4de3-be08-4f17-99f4-8bdec6add915 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:07:26.629526 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d4de3-be08-4f17-99f4-8bdec6add915 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:07:26.629793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d4de3-be08-4f17-99f4-8bdec6add915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:07:26.631823 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a94d4de3-be08-4f17-99f4-8bdec6add915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:07:26.720446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d4de3-be08-4f17-99f4-8bdec6add915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:07:26.721871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d4de3-be08-4f17-99f4-8bdec6add915 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:07:26.743506 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1405/2799] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:07:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:07:27.927691 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fbe0a-c8b9-4243-9175-fd8f95f23165 service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:07:27.929471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ed1fbe0a-c8b9-4243-9175-fd8f95f23165 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:07:27.941171 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fbe0a-c8b9-4243-9175-fd8f95f23165 service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:07:27.945665 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58e4e6-d481-4fb5-aeae-d50642167b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:07:27.946322 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58e4e6-d481-4fb5-aeae-d50642167b4b service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:07:27.946665 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58e4e6-d481-4fb5-aeae-d50642167b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:07:27.947042 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58e4e6-d481-4fb5-aeae-d50642167b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:07:27.948975 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-5a58e4e6-d481-4fb5-aeae-d50642167b4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:07:27.963320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58e4e6-d481-4fb5-aeae-d50642167b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:07:27.964721 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58e4e6-d481-4fb5-aeae-d50642167b4b service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:07:27.976510 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1395/2800] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:07:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:07:28.617013 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d4de3-be08-4f17-99f4-8bdec6add915 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:07:28.618836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a94d4de3-be08-4f17-99f4-8bdec6add915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:07:28.635508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d4de3-be08-4f17-99f4-8bdec6add915 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:07:28.641325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc83eff-9677-4f06-b251-80fc07a855d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:07:28.641962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc83eff-9677-4f06-b251-80fc07a855d0 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:07:28.642248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc83eff-9677-4f06-b251-80fc07a855d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:07:28.642524 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc83eff-9677-4f06-b251-80fc07a855d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:07:28.644441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-0fc83eff-9677-4f06-b251-80fc07a855d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:07:28.671651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc83eff-9677-4f06-b251-80fc07a855d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:07:28.673454 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc83eff-9677-4f06-b251-80fc07a855d0 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:07:28.688233 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1406/2801] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:07:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:07:37.971054 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:07:38.681120 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=56 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:08:28.116164 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58e4e6-d481-4fb5-aeae-d50642167b4b service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:08:28.117871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-5a58e4e6-d481-4fb5-aeae-d50642167b4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:08:28.129278 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58e4e6-d481-4fb5-aeae-d50642167b4b service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:08:28.133704 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbdeb5-219f-40ec-ac53-686b7dbc289a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:08:28.134306 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbdeb5-219f-40ec-ac53-686b7dbc289a service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:08:28.134587 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbdeb5-219f-40ec-ac53-686b7dbc289a service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:08:28.134861 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbdeb5-219f-40ec-ac53-686b7dbc289a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:08:28.143506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbdeb5-219f-40ec-ac53-686b7dbc289a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:08:28.144917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbdeb5-219f-40ec-ac53-686b7dbc289a service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:08:28.154018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1396/2802] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:08:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:08:32.407269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc83eff-9677-4f06-b251-80fc07a855d0 service nova] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:08:32.408988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-0fc83eff-9677-4f06-b251-80fc07a855d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:08:32.422844 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc83eff-9677-4f06-b251-80fc07a855d0 service nova] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:08:32.432817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb50a8-7acc-46c0-9ea5-bcc34764985e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:08:32.432817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb50a8-7acc-46c0-9ea5-bcc34764985e service placement] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:08:32.432817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb50a8-7acc-46c0-9ea5-bcc34764985e service placement] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:08:32.432817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb50a8-7acc-46c0-9ea5-bcc34764985e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:08:32.438730 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb50a8-7acc-46c0-9ea5-bcc34764985e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:08:32.440327 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb50a8-7acc-46c0-9ea5-bcc34764985e service placement] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:08:32.451057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1407/2803] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:08:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:08:38.149843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:08:42.446650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=6 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:11:25.814739 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbdeb5-219f-40ec-ac53-686b7dbc289a service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:11:25.816359 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-7ecbdeb5-219f-40ec-ac53-686b7dbc289a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:11:25.896829 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbdeb5-219f-40ec-ac53-686b7dbc289a service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:11:25.900834 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffaa50-150e-4717-b294-464ca252eed2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:11:25.901442 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffaa50-150e-4717-b294-464ca252eed2 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:11:25.901713 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffaa50-150e-4717-b294-464ca252eed2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:11:25.901984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffaa50-150e-4717-b294-464ca252eed2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:11:25.903841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-33ffaa50-150e-4717-b294-464ca252eed2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:11:25.985953 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffaa50-150e-4717-b294-464ca252eed2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:11:25.987428 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffaa50-150e-4717-b294-464ca252eed2 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:11:26.005948 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1397/2804] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:11:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:11:36.003437 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=70 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:12:00.262990 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb50a8-7acc-46c0-9ea5-bcc34764985e service placement] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:00.272914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb50a8-7acc-46c0-9ea5-bcc34764985e service placement] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:00.276879 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e49237-89a8-43b6-a3c4-f41ded1f438e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:00.277517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e49237-89a8-43b6-a3c4-f41ded1f438e service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:00.277800 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e49237-89a8-43b6-a3c4-f41ded1f438e service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:00.278077 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e49237-89a8-43b6-a3c4-f41ded1f438e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:00.279980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-03e49237-89a8-43b6-a3c4-f41ded1f438e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:00.365854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e49237-89a8-43b6-a3c4-f41ded1f438e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:00.367326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e49237-89a8-43b6-a3c4-f41ded1f438e service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:00.386384 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1408/2805] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:12:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:12:01.128471 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffaa50-150e-4717-b294-464ca252eed2 service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:01.140561 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffaa50-150e-4717-b294-464ca252eed2 service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:01.145679 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef19d39-93cb-401b-a3b6-c584076cac71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:01.146307 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef19d39-93cb-401b-a3b6-c584076cac71 service placement] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:01.146581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef19d39-93cb-401b-a3b6-c584076cac71 service placement] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:01.146858 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef19d39-93cb-401b-a3b6-c584076cac71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:01.148641 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-cef19d39-93cb-401b-a3b6-c584076cac71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:01.161258 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef19d39-93cb-401b-a3b6-c584076cac71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:01.163030 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef19d39-93cb-401b-a3b6-c584076cac71 service placement] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:01.174412 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1398/2806] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:12:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:12:01.234851 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e49237-89a8-43b6-a3c4-f41ded1f438e service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:01.237253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-03e49237-89a8-43b6-a3c4-f41ded1f438e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:01.249428 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e49237-89a8-43b6-a3c4-f41ded1f438e service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:01.254379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3d624-31b5-47b9-bc0a-897a6976a654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:01.255059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3d624-31b5-47b9-bc0a-897a6976a654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:01.255405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3d624-31b5-47b9-bc0a-897a6976a654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:01.255579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3d624-31b5-47b9-bc0a-897a6976a654 admin admin] PATH_INFO: `/v3/users/c8214c2eb78f404d94bfc73ca7139e78` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:01.258170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3d624-31b5-47b9-bc0a-897a6976a654 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8214c2eb78f404d94bfc73ca7139e78)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:01.258627 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3d624-31b5-47b9-bc0a-897a6976a654 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:01.397090 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c9a3d624-31b5-47b9-bc0a-897a6976a654 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8214c2eb78f404d94bfc73ca7139e78'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:12:01.410537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c9a3d624-31b5-47b9-bc0a-897a6976a654 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8214c2eb78f404d94bfc73ca7139e78'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:12:01.473092 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c9a3d624-31b5-47b9-bc0a-897a6976a654 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8214c2eb78f404d94bfc73ca7139e78'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:12:01.480644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c9a3d624-31b5-47b9-bc0a-897a6976a654 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8214c2eb78f404d94bfc73ca7139e78'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:12:01.481296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c9a3d624-31b5-47b9-bc0a-897a6976a654 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8214c2eb78f404d94bfc73ca7139e78'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:12:01.487369 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1409/2807] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 20:12:01 2022] DELETE /identity/v3/users/c8214c2eb78f404d94bfc73ca7139e78 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:12:01.966076 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef19d39-93cb-401b-a3b6-c584076cac71 service placement] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:01.981841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef19d39-93cb-401b-a3b6-c584076cac71 service placement] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:02.046044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00cc18-ec15-4873-853c-95a282633556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:02.046708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00cc18-ec15-4873-853c-95a282633556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:02.047019 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00cc18-ec15-4873-853c-95a282633556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:02.047297 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00cc18-ec15-4873-853c-95a282633556 admin admin] PATH_INFO: `/v3/projects/3374fda464574c07a137d5fe03b02712` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:02.048415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00cc18-ec15-4873-853c-95a282633556 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3374fda464574c07a137d5fe03b02712)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:02.050924 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00cc18-ec15-4873-853c-95a282633556 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:02.160577 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0f00cc18-ec15-4873-853c-95a282633556 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3374fda464574c07a137d5fe03b02712', 'request_id': 'req-0f00cc18-ec15-4873-853c-95a282633556'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:12:02.166939 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0f00cc18-ec15-4873-853c-95a282633556 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3374fda464574c07a137d5fe03b02712', 'request_id': 'req-0f00cc18-ec15-4873-853c-95a282633556'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:12:02.173409 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0f00cc18-ec15-4873-853c-95a282633556 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3374fda464574c07a137d5fe03b02712', 'request_id': 'req-0f00cc18-ec15-4873-853c-95a282633556'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:12:02.243746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0f00cc18-ec15-4873-853c-95a282633556 admin admin] The token cache is being invalidate because project 3374fda464574c07a137d5fe03b02712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 20:12:02.244040 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0f00cc18-ec15-4873-853c-95a282633556 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:12:02.244770 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0f00cc18-ec15-4873-853c-95a282633556 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:12:02.246260 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1399/2808] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 20:12:01 2022] DELETE /identity/v3/projects/3374fda464574c07a137d5fe03b02712 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:12:02.252325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1c718-8e0e-4ec5-864a-c0c16192ae68 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:02.252618 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1c718-8e0e-4ec5-864a-c0c16192ae68 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:02.252899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1c718-8e0e-4ec5-864a-c0c16192ae68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:02.310743 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:12:02.313782 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-89b1c718-8e0e-4ec5-864a-c0c16192ae68 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:12:02.315047 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-89b1c718-8e0e-4ec5-864a-c0c16192ae68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:02.375451 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1410/2809] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:12:02.455541 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd8167-d60d-4eba-849b-0a3de0db7726 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:02.455750 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd8167-d60d-4eba-849b-0a3de0db7726 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:02.456033 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd8167-d60d-4eba-849b-0a3de0db7726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:02.476857 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-05dd8167-d60d-4eba-849b-0a3de0db7726 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:12:02.478243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-05dd8167-d60d-4eba-849b-0a3de0db7726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:02.486377 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1400/2810] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:12:02.490723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1c718-8e0e-4ec5-864a-c0c16192ae68 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:02.500704 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1c718-8e0e-4ec5-864a-c0c16192ae68 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:02.506444 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7bed0-1ae0-4965-ab48-a00c4480b111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:02.507097 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7bed0-1ae0-4965-ab48-a00c4480b111 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:02.507405 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7bed0-1ae0-4965-ab48-a00c4480b111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:02.507685 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7bed0-1ae0-4965-ab48-a00c4480b111 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:02.508345 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7bed0-1ae0-4965-ab48-a00c4480b111 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 20:12:02.508782 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7bed0-1ae0-4965-ab48-a00c4480b111 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:02.510395 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7bed0-1ae0-4965-ab48-a00c4480b111 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:02.523107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1411/2811] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 20:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 20:12:02.527450 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd8167-d60d-4eba-849b-0a3de0db7726 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:02.537693 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd8167-d60d-4eba-849b-0a3de0db7726 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:02.542896 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8208a-456c-4037-a7e0-c8a5243ab683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:02.543614 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8208a-456c-4037-a7e0-c8a5243ab683 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:02.543899 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8208a-456c-4037-a7e0-c8a5243ab683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:02.544208 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8208a-456c-4037-a7e0-c8a5243ab683 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:02.544877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c8208a-456c-4037-a7e0-c8a5243ab683 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:02.546758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c8208a-456c-4037-a7e0-c8a5243ab683 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:02.640654 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1401/2812] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 20:12:02 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:12:02.645650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7bed0-1ae0-4965-ab48-a00c4480b111 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:02.655746 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7bed0-1ae0-4965-ab48-a00c4480b111 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:02.712213 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29a0db-fc85-4713-a4b0-063e56f01a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:02.712571 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29a0db-fc85-4713-a4b0-063e56f01a3c admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:02.712854 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29a0db-fc85-4713-a4b0-063e56f01a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:02.713127 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29a0db-fc85-4713-a4b0-063e56f01a3c admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:02.713752 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29a0db-fc85-4713-a4b0-063e56f01a3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:02.715689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29a0db-fc85-4713-a4b0-063e56f01a3c admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:02.775885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1412/2813] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 20:12:02 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:12:02.780156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8208a-456c-4037-a7e0-c8a5243ab683 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:02.791426 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8208a-456c-4037-a7e0-c8a5243ab683 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:02.796031 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1a667-7ddc-4d52-a031-1a22758b7c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:02.796626 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1a667-7ddc-4d52-a031-1a22758b7c81 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:02.796908 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1a667-7ddc-4d52-a031-1a22758b7c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:02.797186 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1a667-7ddc-4d52-a031-1a22758b7c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:02.797828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1a667-7ddc-4d52-a031-1a22758b7c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:02.800097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1a667-7ddc-4d52-a031-1a22758b7c81 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:02.810414 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1402/2814] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:12:02.814474 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29a0db-fc85-4713-a4b0-063e56f01a3c admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:02.825258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29a0db-fc85-4713-a4b0-063e56f01a3c admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:02.830057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e05525-2030-4649-924e-5edab362fc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:02.833371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e05525-2030-4649-924e-5edab362fc5c admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:02.833371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e05525-2030-4649-924e-5edab362fc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:02.833371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e05525-2030-4649-924e-5edab362fc5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:02.833371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e05525-2030-4649-924e-5edab362fc5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:02.833688 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e05525-2030-4649-924e-5edab362fc5c admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:02.844795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1413/2815] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:12:02.849486 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1a667-7ddc-4d52-a031-1a22758b7c81 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:02.859787 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1a667-7ddc-4d52-a031-1a22758b7c81 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:02.865532 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd280b9-dbbb-4781-b0ec-1c780a5e61fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:02.866182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd280b9-dbbb-4781-b0ec-1c780a5e61fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:02.866465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd280b9-dbbb-4781-b0ec-1c780a5e61fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:02.866752 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd280b9-dbbb-4781-b0ec-1c780a5e61fc admin admin] PATH_INFO: `/v3/projects/21718c5f45b74854bd0a487939a0bd23/users/7b585708ea6446e398c45196e9a992ac/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:02.883192 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd280b9-dbbb-4781-b0ec-1c780a5e61fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=21718c5f45b74854bd0a487939a0bd23, user_id=7b585708ea6446e398c45196e9a992ac, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:02.886542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd280b9-dbbb-4781-b0ec-1c780a5e61fc admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:02.942181 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1403/2816] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:12:02 2022] PUT /identity/v3/projects/21718c5f45b74854bd0a487939a0bd23/users/7b585708ea6446e398c45196e9a992ac/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:12:02.947446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e110ad-274f-4023-9384-ee83c1add4cc None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:02.947734 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e110ad-274f-4023-9384-ee83c1add4cc None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:02.948015 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e110ad-274f-4023-9384-ee83c1add4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:02.967115 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a3e110ad-274f-4023-9384-ee83c1add4cc None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:12:02.968252 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a3e110ad-274f-4023-9384-ee83c1add4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:03.031633 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1414/2817] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:12:03.038684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd280b9-dbbb-4781-b0ec-1c780a5e61fc admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:03.040560 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-cbd280b9-dbbb-4781-b0ec-1c780a5e61fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:03.047213 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:12:03.101801 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd280b9-dbbb-4781-b0ec-1c780a5e61fc admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:03.105626 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb3786-da4a-481b-b5eb-9970c01f2618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:03.108695 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb3786-da4a-481b-b5eb-9970c01f2618 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:03.108695 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb3786-da4a-481b-b5eb-9970c01f2618 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:03.108695 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb3786-da4a-481b-b5eb-9970c01f2618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:03.116631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb3786-da4a-481b-b5eb-9970c01f2618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:03.118050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb3786-da4a-481b-b5eb-9970c01f2618 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:03.128468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1404/2818] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:12:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:12:08.356194 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7b272-d8d5-4ad2-aae3-0f4251c2bb4e None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:08.356664 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7b272-d8d5-4ad2-aae3-0f4251c2bb4e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:08.356752 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7b272-d8d5-4ad2-aae3-0f4251c2bb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:08.385859 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:12:08.397378 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-8ae7b272-d8d5-4ad2-aae3-0f4251c2bb4e None None] MFA Rules not processed for user `7b585708ea6446e398c45196e9a992ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:12:08.401154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-8ae7b272-d8d5-4ad2-aae3-0f4251c2bb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:08.473962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1415/2819] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:12:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:12:08.478732 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb3786-da4a-481b-b5eb-9970c01f2618 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:08.488929 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb3786-da4a-481b-b5eb-9970c01f2618 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:08.494102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54237e-1134-4251-b5cc-2f82cbd62e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:08.494746 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54237e-1134-4251-b5cc-2f82cbd62e9f admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:08.495071 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54237e-1134-4251-b5cc-2f82cbd62e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:08.495348 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54237e-1134-4251-b5cc-2f82cbd62e9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:08.495999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54237e-1134-4251-b5cc-2f82cbd62e9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:08.497972 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54237e-1134-4251-b5cc-2f82cbd62e9f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:08.578207 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1405/2820] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 20:12:08 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:12:08.583760 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7b272-d8d5-4ad2-aae3-0f4251c2bb4e None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:08.596319 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7b272-d8d5-4ad2-aae3-0f4251c2bb4e None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:08.657334 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7bd73-1995-4222-ba22-e34037b93758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:08.658778 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7bd73-1995-4222-ba22-e34037b93758 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:08.659203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7bd73-1995-4222-ba22-e34037b93758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:08.659582 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7bd73-1995-4222-ba22-e34037b93758 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:08.660236 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7bd73-1995-4222-ba22-e34037b93758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:08.662279 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7bd73-1995-4222-ba22-e34037b93758 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:08.743258 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1416/2821] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 20:12:08 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:12:08.747601 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54237e-1134-4251-b5cc-2f82cbd62e9f admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:08.759152 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54237e-1134-4251-b5cc-2f82cbd62e9f admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:08.763989 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45f051-a2dd-49d1-86c2-747d849e331a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:08.764638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45f051-a2dd-49d1-86c2-747d849e331a admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:08.764916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45f051-a2dd-49d1-86c2-747d849e331a admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:08.765188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45f051-a2dd-49d1-86c2-747d849e331a admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:08.765876 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45f051-a2dd-49d1-86c2-747d849e331a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:08.767527 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45f051-a2dd-49d1-86c2-747d849e331a admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:08.779549 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1406/2822] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:12:08.784781 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7bd73-1995-4222-ba22-e34037b93758 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:08.795488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7bd73-1995-4222-ba22-e34037b93758 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:08.802088 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab14b3e-5592-456b-9481-89883c707422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:08.802827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab14b3e-5592-456b-9481-89883c707422 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:08.803152 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab14b3e-5592-456b-9481-89883c707422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:08.803428 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab14b3e-5592-456b-9481-89883c707422 admin admin] PATH_INFO: `/v3/projects/65ce797d4abf45ee9abccf740960d9a0/users/6eecc2b3f85640e69d866efede43f74c/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:08.820066 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab14b3e-5592-456b-9481-89883c707422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65ce797d4abf45ee9abccf740960d9a0, user_id=6eecc2b3f85640e69d866efede43f74c, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:08.822715 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab14b3e-5592-456b-9481-89883c707422 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:08.909040 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1417/2823] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:12:08 2022] PUT /identity/v3/projects/65ce797d4abf45ee9abccf740960d9a0/users/6eecc2b3f85640e69d866efede43f74c/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:12:08.913438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45f051-a2dd-49d1-86c2-747d849e331a admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:08.923956 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45f051-a2dd-49d1-86c2-747d849e331a admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:08.986217 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5dc74a-ed9f-4183-9dba-41f3c67dec8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:08.986947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5dc74a-ed9f-4183-9dba-41f3c67dec8b admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:08.987301 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5dc74a-ed9f-4183-9dba-41f3c67dec8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:08.987602 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5dc74a-ed9f-4183-9dba-41f3c67dec8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:08.988310 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5dc74a-ed9f-4183-9dba-41f3c67dec8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:08.989919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5dc74a-ed9f-4183-9dba-41f3c67dec8b admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:09.000916 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1407/2824] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:12:09.005301 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab14b3e-5592-456b-9481-89883c707422 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:09.019723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab14b3e-5592-456b-9481-89883c707422 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:09.024494 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4f6eb-3eac-492b-8a0b-1a955cd1f39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:09.025166 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4f6eb-3eac-492b-8a0b-1a955cd1f39b admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:09.025446 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4f6eb-3eac-492b-8a0b-1a955cd1f39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:09.025743 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4f6eb-3eac-492b-8a0b-1a955cd1f39b admin admin] PATH_INFO: `/v3/projects/65ce797d4abf45ee9abccf740960d9a0/users/6eecc2b3f85640e69d866efede43f74c/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:09.027673 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4f6eb-3eac-492b-8a0b-1a955cd1f39b admin admin] RBAC: Authorizing `identity:create_grant(project_id=65ce797d4abf45ee9abccf740960d9a0, user_id=6eecc2b3f85640e69d866efede43f74c, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:09.030436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4f6eb-3eac-492b-8a0b-1a955cd1f39b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:09.042219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1418/2825] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:12:09 2022] PUT /identity/v3/projects/65ce797d4abf45ee9abccf740960d9a0/users/6eecc2b3f85640e69d866efede43f74c/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:12:14.425551 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3efae-5cba-436f-982e-e5d278d9bc1e None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:14.426206 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3efae-5cba-436f-982e-e5d278d9bc1e None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:14.426206 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3efae-5cba-436f-982e-e5d278d9bc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:14.456510 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-e3e3efae-5cba-436f-982e-e5d278d9bc1e None None] MFA Rules not processed for user `6eecc2b3f85640e69d866efede43f74c`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:12:14.457669 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e3e3efae-5cba-436f-982e-e5d278d9bc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:14.537631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:12:14.548138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1408/2826] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:12:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:12:14.588020 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4f6eb-3eac-492b-8a0b-1a955cd1f39b admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:14.590022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-1ab4f6eb-3eac-492b-8a0b-1a955cd1f39b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:14.658323 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:12:14.679372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4f6eb-3eac-492b-8a0b-1a955cd1f39b admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:14.686231 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397775eb-d828-4b90-935f-43c5d13f6696 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:14.686861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397775eb-d828-4b90-935f-43c5d13f6696 service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:14.687204 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397775eb-d828-4b90-935f-43c5d13f6696 service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:14.691045 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397775eb-d828-4b90-935f-43c5d13f6696 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:14.770039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397775eb-d828-4b90-935f-43c5d13f6696 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:14.773129 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397775eb-d828-4b90-935f-43c5d13f6696 service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:14.784813 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1419/2827] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:12:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:12:17.103022 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c28a5e-0927-4de2-aedf-91b7df95917f None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:17.103519 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c28a5e-0927-4de2-aedf-91b7df95917f None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:17.103597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c28a5e-0927-4de2-aedf-91b7df95917f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:17.124190 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-40c28a5e-0927-4de2-aedf-91b7df95917f None None] MFA Rules not processed for user `7b585708ea6446e398c45196e9a992ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:12:17.125405 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-40c28a5e-0927-4de2-aedf-91b7df95917f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:17.133003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1409/2828] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:12:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:12:24.780497 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=31 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:12:27.118602 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=5 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:12:41.141282 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397775eb-d828-4b90-935f-43c5d13f6696 service nova] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:41.142745 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-397775eb-d828-4b90-935f-43c5d13f6696 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:41.213363 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397775eb-d828-4b90-935f-43c5d13f6696 service nova] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:41.217083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad0e50-a599-4253-907f-20a7c7aed135 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:41.217724 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad0e50-a599-4253-907f-20a7c7aed135 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:41.218005 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad0e50-a599-4253-907f-20a7c7aed135 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:41.218282 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad0e50-a599-4253-907f-20a7c7aed135 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:41.220154 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-01ad0e50-a599-4253-907f-20a7c7aed135 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:41.234022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad0e50-a599-4253-907f-20a7c7aed135 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:41.235490 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad0e50-a599-4253-907f-20a7c7aed135 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:41.247783 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1420/2829] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:12:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:12:41.537456 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c28a5e-0927-4de2-aedf-91b7df95917f None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:41.592824 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c28a5e-0927-4de2-aedf-91b7df95917f None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:41.596472 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84571f51-7da9-4d0d-ab15-aebc3d368cc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:41.597020 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84571f51-7da9-4d0d-ab15-aebc3d368cc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:41.597303 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84571f51-7da9-4d0d-ab15-aebc3d368cc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:41.597581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84571f51-7da9-4d0d-ab15-aebc3d368cc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:41.599934 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-84571f51-7da9-4d0d-ab15-aebc3d368cc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:41.609410 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84571f51-7da9-4d0d-ab15-aebc3d368cc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:41.610828 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84571f51-7da9-4d0d-ab15-aebc3d368cc5 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:41.619392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1410/2830] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:12:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:12:42.249760 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad0e50-a599-4253-907f-20a7c7aed135 service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:42.254712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-01ad0e50-a599-4253-907f-20a7c7aed135 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:42.264618 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84571f51-7da9-4d0d-ab15-aebc3d368cc5 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:42.266171 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-84571f51-7da9-4d0d-ab15-aebc3d368cc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:42.278291 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84571f51-7da9-4d0d-ab15-aebc3d368cc5 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:42.282003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb6fe6-dc00-4ebf-b1fb-486fb9db265d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:42.282614 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb6fe6-dc00-4ebf-b1fb-486fb9db265d service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:42.282993 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb6fe6-dc00-4ebf-b1fb-486fb9db265d service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:42.283282 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb6fe6-dc00-4ebf-b1fb-486fb9db265d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:42.285035 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b6cb6fe6-dc00-4ebf-b1fb-486fb9db265d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:42.300392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb6fe6-dc00-4ebf-b1fb-486fb9db265d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:42.302067 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb6fe6-dc00-4ebf-b1fb-486fb9db265d service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:42.313630 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1411/2831] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:12:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 20:12:42.328483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad0e50-a599-4253-907f-20a7c7aed135 service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:42.334168 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf99d89-e507-46a3-bb5b-0b5344d9afd7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:42.334796 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf99d89-e507-46a3-bb5b-0b5344d9afd7 service placement] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:42.335112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf99d89-e507-46a3-bb5b-0b5344d9afd7 service placement] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:42.335391 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf99d89-e507-46a3-bb5b-0b5344d9afd7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:42.337170 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ecf99d89-e507-46a3-bb5b-0b5344d9afd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:42.350926 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf99d89-e507-46a3-bb5b-0b5344d9afd7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:42.352527 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf99d89-e507-46a3-bb5b-0b5344d9afd7 service placement] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:42.363105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1421/2832] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:12:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:12:45.315897 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb6fe6-dc00-4ebf-b1fb-486fb9db265d service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:45.326673 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb6fe6-dc00-4ebf-b1fb-486fb9db265d service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:45.334171 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fdb05-fd9b-4502-ae26-c9bc5ebe9a12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:45.334171 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fdb05-fd9b-4502-ae26-c9bc5ebe9a12 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:45.334171 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fdb05-fd9b-4502-ae26-c9bc5ebe9a12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:45.334171 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fdb05-fd9b-4502-ae26-c9bc5ebe9a12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:45.334512 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-aa1fdb05-fd9b-4502-ae26-c9bc5ebe9a12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:45.344050 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fdb05-fd9b-4502-ae26-c9bc5ebe9a12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:45.345639 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fdb05-fd9b-4502-ae26-c9bc5ebe9a12 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:45.354778 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1412/2833] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:12:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:12:46.123018 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf99d89-e507-46a3-bb5b-0b5344d9afd7 service placement] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:46.137171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf99d89-e507-46a3-bb5b-0b5344d9afd7 service placement] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:46.142722 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39304b8-5c67-4a6c-9776-32371e9e346e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:46.143362 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39304b8-5c67-4a6c-9776-32371e9e346e service nova] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:46.143645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39304b8-5c67-4a6c-9776-32371e9e346e service nova] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:46.144085 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39304b8-5c67-4a6c-9776-32371e9e346e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:46.145927 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c39304b8-5c67-4a6c-9776-32371e9e346e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:46.158485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39304b8-5c67-4a6c-9776-32371e9e346e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:46.160075 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39304b8-5c67-4a6c-9776-32371e9e346e service nova] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:46.175061 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1422/2834] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:12:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:12:55.350386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=38 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:12:56.169814 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=64 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:12:58.674594 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fdb05-fd9b-4502-ae26-c9bc5ebe9a12 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:12:58.689637 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fdb05-fd9b-4502-ae26-c9bc5ebe9a12 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:12:58.695603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b3ee5-4b00-4400-b572-78ac2e1ccbd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:12:58.696421 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b3ee5-4b00-4400-b572-78ac2e1ccbd9 service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:12:58.696821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b3ee5-4b00-4400-b572-78ac2e1ccbd9 service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:12:58.697104 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b3ee5-4b00-4400-b572-78ac2e1ccbd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:12:58.698910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-031b3ee5-4b00-4400-b572-78ac2e1ccbd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:12:58.712248 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b3ee5-4b00-4400-b572-78ac2e1ccbd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:12:58.713824 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b3ee5-4b00-4400-b572-78ac2e1ccbd9 service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:12:58.725969 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1413/2835] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:12:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:13:02.483047 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39304b8-5c67-4a6c-9776-32371e9e346e service nova] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:13:02.493480 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39304b8-5c67-4a6c-9776-32371e9e346e service nova] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:13:02.497254 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a862fba-7fad-49bc-b865-4a0738731825 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:13:02.497855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a862fba-7fad-49bc-b865-4a0738731825 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:13:02.498139 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a862fba-7fad-49bc-b865-4a0738731825 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:13:02.498416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a862fba-7fad-49bc-b865-4a0738731825 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:13:02.507980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a862fba-7fad-49bc-b865-4a0738731825 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:13:02.509410 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a862fba-7fad-49bc-b865-4a0738731825 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:13:02.519065 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1423/2836] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:13:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:13:04.709172 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b3ee5-4b00-4400-b572-78ac2e1ccbd9 service nova] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:13:04.722079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b3ee5-4b00-4400-b572-78ac2e1ccbd9 service nova] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:13:04.728366 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a2bea-ba23-49e7-9e91-6e7769d857bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd461422ce98442bcbe6b254a8dad5e8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:13:04.729095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a2bea-ba23-49e7-9e91-6e7769d857bd service nova] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:13:04.729379 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a2bea-ba23-49e7-9e91-6e7769d857bd service nova] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:13:04.729658 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a2bea-ba23-49e7-9e91-6e7769d857bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:13:04.731442 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-637a2bea-ba23-49e7-9e91-6e7769d857bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:13:04.744795 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a2bea-ba23-49e7-9e91-6e7769d857bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:13:04.746372 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a2bea-ba23-49e7-9e91-6e7769d857bd service nova] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:13:04.758261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1414/2837] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:13:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:13:12.514797 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=6 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:13:14.752771 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=12 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:13:27.483919 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a862fba-7fad-49bc-b865-4a0738731825 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:13:27.485320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4a862fba-7fad-49bc-b865-4a0738731825 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:13:27.497594 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a862fba-7fad-49bc-b865-4a0738731825 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:13:27.501174 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd7d26-d82f-4fcf-a1c7-91f09f688517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:13:27.504316 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd7d26-d82f-4fcf-a1c7-91f09f688517 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:13:27.504316 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd7d26-d82f-4fcf-a1c7-91f09f688517 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:13:27.504316 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd7d26-d82f-4fcf-a1c7-91f09f688517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:13:27.504741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-2efd7d26-d82f-4fcf-a1c7-91f09f688517 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:13:27.587483 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd7d26-d82f-4fcf-a1c7-91f09f688517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:13:27.588930 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd7d26-d82f-4fcf-a1c7-91f09f688517 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:13:27.606804 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1424/2838] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:13:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:13:35.402418 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a2bea-ba23-49e7-9e91-6e7769d857bd service nova] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:13:35.466079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a2bea-ba23-49e7-9e91-6e7769d857bd service nova] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:13:35.471445 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95882f17-ae4c-4bef-98ef-f0164190f535 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:13:35.472049 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95882f17-ae4c-4bef-98ef-f0164190f535 service placement] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:13:35.473318 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95882f17-ae4c-4bef-98ef-f0164190f535 service placement] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:13:35.473780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95882f17-ae4c-4bef-98ef-f0164190f535 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:13:35.475579 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-95882f17-ae4c-4bef-98ef-f0164190f535 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:13:35.509914 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95882f17-ae4c-4bef-98ef-f0164190f535 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:13:35.511590 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95882f17-ae4c-4bef-98ef-f0164190f535 service placement] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:13:35.521684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1415/2839] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:13:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:13:37.604255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=41 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:13:45.518001 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=36 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:15:33.760979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45cd24-178b-44a8-889a-d24b6472d4b8 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:15:33.762987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45cd24-178b-44a8-889a-d24b6472d4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:15:33.762987 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45cd24-178b-44a8-889a-d24b6472d4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:15:33.828118 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-4d45cd24-178b-44a8-889a-d24b6472d4b8 None None] MFA Rules not processed for user `b7a51545146846cc8284b7044b32198a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:15:33.829690 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4d45cd24-178b-44a8-889a-d24b6472d4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:15:33.837346 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1425/2840] 10.222.0.59 () {58 vars in 1062 bytes} [Wed Aug 31 20:15:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:15:33.843478 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95882f17-ae4c-4bef-98ef-f0164190f535 service placement] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:15:33.852880 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95882f17-ae4c-4bef-98ef-f0164190f535 service placement] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:15:33.856561 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a832e3d-dc9d-4411-9f05-10a733d9103e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:15:33.857174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a832e3d-dc9d-4411-9f05-10a733d9103e service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:15:33.857446 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a832e3d-dc9d-4411-9f05-10a733d9103e service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:15:33.857718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a832e3d-dc9d-4411-9f05-10a733d9103e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:15:33.859497 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-2a832e3d-dc9d-4411-9f05-10a733d9103e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:15:33.869028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a832e3d-dc9d-4411-9f05-10a733d9103e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:15:33.870425 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a832e3d-dc9d-4411-9f05-10a733d9103e service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:15:33.878249 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1416/2841] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:15:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 20:15:33.963197 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45cd24-178b-44a8-889a-d24b6472d4b8 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:15:33.971427 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45cd24-178b-44a8-889a-d24b6472d4b8 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:15:33.975634 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae4d31-1df4-4f5d-b72a-e6a718c21467 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:15:33.976225 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae4d31-1df4-4f5d-b72a-e6a718c21467 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:15:33.976503 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae4d31-1df4-4f5d-b72a-e6a718c21467 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:15:33.976772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae4d31-1df4-4f5d-b72a-e6a718c21467 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:15:33.978470 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-86ae4d31-1df4-4f5d-b72a-e6a718c21467 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:15:33.992017 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae4d31-1df4-4f5d-b72a-e6a718c21467 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:15:33.993442 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae4d31-1df4-4f5d-b72a-e6a718c21467 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:15:34.004217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1426/2842] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:15:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:15:34.059204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cbe7d5-6c9b-4182-aebf-2b44ba3b8442 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:15:34.059540 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cbe7d5-6c9b-4182-aebf-2b44ba3b8442 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:15:34.059845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cbe7d5-6c9b-4182-aebf-2b44ba3b8442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:15:34.099035 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-20cbe7d5-6c9b-4182-aebf-2b44ba3b8442 None None] MFA Rules not processed for user `eced13cc4030423abe63bb1527f6d35a`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:15:34.100288 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-20cbe7d5-6c9b-4182-aebf-2b44ba3b8442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:15:34.108958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1417/2843] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 31 20:15:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:15:34.121143 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae4d31-1df4-4f5d-b72a-e6a718c21467 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:15:34.135865 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae4d31-1df4-4f5d-b72a-e6a718c21467 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:15:34.144467 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d97df8-f2ea-4fb9-86ea-9e3ad30250cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:15:34.145141 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d97df8-f2ea-4fb9-86ea-9e3ad30250cc service placement] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:15:34.145429 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d97df8-f2ea-4fb9-86ea-9e3ad30250cc service placement] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:15:34.145747 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d97df8-f2ea-4fb9-86ea-9e3ad30250cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:15:34.154320 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d97df8-f2ea-4fb9-86ea-9e3ad30250cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:15:34.155960 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d97df8-f2ea-4fb9-86ea-9e3ad30250cc service placement] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:15:34.166535 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1427/2844] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:15:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:15:44.096758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=15 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:15:44.162048 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=30 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:17:21.964108 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a3123-ab57-4db6-8e26-5dfbf4151a45 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:17:21.964552 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a3123-ab57-4db6-8e26-5dfbf4151a45 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:17:21.964657 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a3123-ab57-4db6-8e26-5dfbf4151a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:17:21.985918 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-7d3a3123-ab57-4db6-8e26-5dfbf4151a45 None None] MFA Rules not processed for user `b7a51545146846cc8284b7044b32198a`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:17:21.987684 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-7d3a3123-ab57-4db6-8e26-5dfbf4151a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:17:21.995099 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1418/2845] 10.222.0.59 () {58 vars in 1062 bytes} [Wed Aug 31 20:17:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:17:22.001434 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d97df8-f2ea-4fb9-86ea-9e3ad30250cc service placement] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:17:22.011783 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d97df8-f2ea-4fb9-86ea-9e3ad30250cc service placement] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:17:22.015441 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20608c-1a6b-4505-8885-d4a7d1244aa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:17:22.016035 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20608c-1a6b-4505-8885-d4a7d1244aa3 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:17:22.016311 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20608c-1a6b-4505-8885-d4a7d1244aa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:17:22.016581 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20608c-1a6b-4505-8885-d4a7d1244aa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:17:22.026502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20608c-1a6b-4505-8885-d4a7d1244aa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:17:22.027975 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20608c-1a6b-4505-8885-d4a7d1244aa3 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:17:22.037613 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1428/2846] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:17:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:17:28.433701 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a3123-ab57-4db6-8e26-5dfbf4151a45 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:17:28.453141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a3123-ab57-4db6-8e26-5dfbf4151a45 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:17:28.456741 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c63452-5b3d-4fe0-9f87-9759bc814e72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:17:28.457353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c63452-5b3d-4fe0-9f87-9759bc814e72 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:17:28.457643 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c63452-5b3d-4fe0-9f87-9759bc814e72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:17:28.457917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c63452-5b3d-4fe0-9f87-9759bc814e72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:17:28.525386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c63452-5b3d-4fe0-9f87-9759bc814e72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:17:28.526908 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c63452-5b3d-4fe0-9f87-9759bc814e72 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:17:28.537668 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1419/2847] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:17:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:17:32.033245 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=6 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:17:32.710897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20608c-1a6b-4505-8885-d4a7d1244aa3 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:17:32.712663 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ff20608c-1a6b-4505-8885-d4a7d1244aa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:17:32.729319 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20608c-1a6b-4505-8885-d4a7d1244aa3 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:17:32.734672 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567a899-21c2-4b8e-a2d5-e11f6bee706c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:17:32.736005 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567a899-21c2-4b8e-a2d5-e11f6bee706c service placement] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:17:32.736005 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567a899-21c2-4b8e-a2d5-e11f6bee706c service placement] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:17:32.736005 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567a899-21c2-4b8e-a2d5-e11f6bee706c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:17:32.737757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-4567a899-21c2-4b8e-a2d5-e11f6bee706c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:17:32.750389 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567a899-21c2-4b8e-a2d5-e11f6bee706c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:17:32.752115 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567a899-21c2-4b8e-a2d5-e11f6bee706c service placement] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:17:32.762946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1429/2848] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:17:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:17:36.226797 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c63452-5b3d-4fe0-9f87-9759bc814e72 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:17:36.239548 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c63452-5b3d-4fe0-9f87-9759bc814e72 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:17:36.246139 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597ffac-2378-4449-8b4f-f9a119930b9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:17:36.250553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597ffac-2378-4449-8b4f-f9a119930b9e service placement] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:17:36.250553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597ffac-2378-4449-8b4f-f9a119930b9e service placement] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:17:36.250553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597ffac-2378-4449-8b4f-f9a119930b9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:17:36.250553 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-6597ffac-2378-4449-8b4f-f9a119930b9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:17:36.262488 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597ffac-2378-4449-8b4f-f9a119930b9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:17:36.264147 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597ffac-2378-4449-8b4f-f9a119930b9e service placement] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:17:36.276174 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1420/2849] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:17:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:17:36.386489 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567a899-21c2-4b8e-a2d5-e11f6bee706c service placement] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:17:36.397543 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567a899-21c2-4b8e-a2d5-e11f6bee706c service placement] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:17:36.403358 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67701e9-997e-49e7-ae6b-88e40a404259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:17:36.404118 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67701e9-997e-49e7-ae6b-88e40a404259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:17:36.404252 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67701e9-997e-49e7-ae6b-88e40a404259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:17:36.404521 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67701e9-997e-49e7-ae6b-88e40a404259 admin admin] PATH_INFO: `/v3/users/7b585708ea6446e398c45196e9a992ac` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:17:36.406060 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67701e9-997e-49e7-ae6b-88e40a404259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b585708ea6446e398c45196e9a992ac)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:17:36.407999 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67701e9-997e-49e7-ae6b-88e40a404259 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:17:36.577816 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e67701e9-997e-49e7-ae6b-88e40a404259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b585708ea6446e398c45196e9a992ac'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:36.591196 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e67701e9-997e-49e7-ae6b-88e40a404259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b585708ea6446e398c45196e9a992ac'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:36.665219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e67701e9-997e-49e7-ae6b-88e40a404259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b585708ea6446e398c45196e9a992ac'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:36.673132 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e67701e9-997e-49e7-ae6b-88e40a404259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b585708ea6446e398c45196e9a992ac'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:36.673785 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e67701e9-997e-49e7-ae6b-88e40a404259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b585708ea6446e398c45196e9a992ac'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:36.679693 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1430/2850] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 20:17:36 2022] DELETE /identity/v3/users/7b585708ea6446e398c45196e9a992ac => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:17:36.979261 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597ffac-2378-4449-8b4f-f9a119930b9e service placement] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:17:36.991129 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597ffac-2378-4449-8b4f-f9a119930b9e service placement] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:17:37.051773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:17:37.052407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:17:37.052688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:17:37.052963 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62 admin admin] PATH_INFO: `/v3/projects/21718c5f45b74854bd0a487939a0bd23` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:17:37.054074 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21718c5f45b74854bd0a487939a0bd23)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:17:37.056913 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:17:37.149117 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21718c5f45b74854bd0a487939a0bd23', 'request_id': 'req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:37.154767 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21718c5f45b74854bd0a487939a0bd23', 'request_id': 'req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:37.161082 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21718c5f45b74854bd0a487939a0bd23', 'request_id': 'req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:37.165316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:17:37.217122 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62 admin admin] The token cache is being invalidate because project 21718c5f45b74854bd0a487939a0bd23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 20:17:37.217413 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:37.218356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:37.219959 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1421/2851] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 20:17:36 2022] DELETE /identity/v3/projects/21718c5f45b74854bd0a487939a0bd23 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 20:17:37.224359 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67701e9-997e-49e7-ae6b-88e40a404259 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:17:37.225703 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-e67701e9-997e-49e7-ae6b-88e40a404259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:17:37.239089 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67701e9-997e-49e7-ae6b-88e40a404259 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:17:37.301239 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0cd6e-51dd-4bd6-8c04-c638d6487e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:17:37.301882 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0cd6e-51dd-4bd6-8c04-c638d6487e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:17:37.302195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0cd6e-51dd-4bd6-8c04-c638d6487e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:17:37.302684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0cd6e-51dd-4bd6-8c04-c638d6487e73 admin admin] PATH_INFO: `/v3/users/6eecc2b3f85640e69d866efede43f74c` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:17:37.303967 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0cd6e-51dd-4bd6-8c04-c638d6487e73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eecc2b3f85640e69d866efede43f74c)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:17:37.306629 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0cd6e-51dd-4bd6-8c04-c638d6487e73 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:17:37.484753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a2f0cd6e-51dd-4bd6-8c04-c638d6487e73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eecc2b3f85640e69d866efede43f74c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:37.488843 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:17:37.491054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a2f0cd6e-51dd-4bd6-8c04-c638d6487e73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eecc2b3f85640e69d866efede43f74c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:37.619643 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a2f0cd6e-51dd-4bd6-8c04-c638d6487e73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eecc2b3f85640e69d866efede43f74c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:37.628182 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a2f0cd6e-51dd-4bd6-8c04-c638d6487e73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eecc2b3f85640e69d866efede43f74c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:37.628988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-a2f0cd6e-51dd-4bd6-8c04-c638d6487e73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eecc2b3f85640e69d866efede43f74c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:37.636004 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1431/2852] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 20:17:37 2022] DELETE /identity/v3/users/6eecc2b3f85640e69d866efede43f74c => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:17:37.941028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:17:37.942525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:17:37.956638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd7a82-7895-4be7-b9ca-027fe5f2ad62 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:17:38.016921 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1921b-c123-442c-b37f-6c9f41d412ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:17:38.017479 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1921b-c123-442c-b37f-6c9f41d412ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:17:38.017763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1921b-c123-442c-b37f-6c9f41d412ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:17:38.018039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1921b-c123-442c-b37f-6c9f41d412ae admin admin] PATH_INFO: `/v3/projects/65ce797d4abf45ee9abccf740960d9a0` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:17:38.019244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1921b-c123-442c-b37f-6c9f41d412ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=65ce797d4abf45ee9abccf740960d9a0)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:17:38.021178 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1921b-c123-442c-b37f-6c9f41d412ae admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:17:38.129919 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-45d1921b-c123-442c-b37f-6c9f41d412ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65ce797d4abf45ee9abccf740960d9a0', 'request_id': 'req-45d1921b-c123-442c-b37f-6c9f41d412ae'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:38.136876 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-45d1921b-c123-442c-b37f-6c9f41d412ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65ce797d4abf45ee9abccf740960d9a0', 'request_id': 'req-45d1921b-c123-442c-b37f-6c9f41d412ae'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:38.143155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-45d1921b-c123-442c-b37f-6c9f41d412ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65ce797d4abf45ee9abccf740960d9a0', 'request_id': 'req-45d1921b-c123-442c-b37f-6c9f41d412ae'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:38.224836 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-45d1921b-c123-442c-b37f-6c9f41d412ae admin admin] The token cache is being invalidate because project 65ce797d4abf45ee9abccf740960d9a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 20:17:38.225135 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-45d1921b-c123-442c-b37f-6c9f41d412ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:38.225763 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-45d1921b-c123-442c-b37f-6c9f41d412ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:17:38.228085 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1422/2853] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 20:17:37 2022] DELETE /identity/v3/projects/65ce797d4abf45ee9abccf740960d9a0 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:17:47.710415 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:17:48.221358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:18:28.567957 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb600a8-ca5e-4e52-8819-64a8c8729be7 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:18:28.568575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb600a8-ca5e-4e52-8819-64a8c8729be7 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:18:28.568575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb600a8-ca5e-4e52-8819-64a8c8729be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:18:28.620035 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-2bb600a8-ca5e-4e52-8819-64a8c8729be7 None None] MFA Rules not processed for user `108a698aa3e6477b9ae896d68f22bd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:18:28.621828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-2bb600a8-ca5e-4e52-8819-64a8c8729be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:18:28.683795 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1432/2854] 10.222.0.59 () {58 vars in 1061 bytes} [Wed Aug 31 20:18:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:18:28.691246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1921b-c123-442c-b37f-6c9f41d412ae admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:18:28.702017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1921b-c123-442c-b37f-6c9f41d412ae admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:18:28.706452 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eeae8-d2d8-4902-8466-64e098242f26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:18:28.707225 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eeae8-d2d8-4902-8466-64e098242f26 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:18:28.707581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eeae8-d2d8-4902-8466-64e098242f26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:18:28.707793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eeae8-d2d8-4902-8466-64e098242f26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:18:28.709572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-7a0eeae8-d2d8-4902-8466-64e098242f26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:18:28.782777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0eeae8-d2d8-4902-8466-64e098242f26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:18:28.784252 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0eeae8-d2d8-4902-8466-64e098242f26 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:18:28.794520 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1423/2855] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:18:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:18:38.676720 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=29 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:18:38.789395 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=28 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:18:41.362804 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb600a8-ca5e-4e52-8819-64a8c8729be7 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:18:41.364348 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-2bb600a8-ca5e-4e52-8819-64a8c8729be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:18:41.442313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb600a8-ca5e-4e52-8819-64a8c8729be7 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:18:41.448874 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa6684-06ca-49b5-96a4-23b5a89df012 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:18:41.449503 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa6684-06ca-49b5-96a4-23b5a89df012 service placement] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:18:41.449791 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa6684-06ca-49b5-96a4-23b5a89df012 service placement] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:18:41.450068 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa6684-06ca-49b5-96a4-23b5a89df012 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:18:41.453135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-78fa6684-06ca-49b5-96a4-23b5a89df012 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:18:41.467619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa6684-06ca-49b5-96a4-23b5a89df012 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:18:41.469304 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa6684-06ca-49b5-96a4-23b5a89df012 service placement] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:18:41.480591 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1433/2856] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:18:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:18:51.475706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=32 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:19:32.977977 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eeae8-d2d8-4902-8466-64e098242f26 service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:32.980403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-7a0eeae8-d2d8-4902-8466-64e098242f26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:33.037312 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eeae8-d2d8-4902-8466-64e098242f26 service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:33.040977 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02abaf24-492a-427e-93b6-9c2b4f080cd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:33.042157 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abaf24-492a-427e-93b6-9c2b4f080cd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:33.042470 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abaf24-492a-427e-93b6-9c2b4f080cd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:33.042965 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abaf24-492a-427e-93b6-9c2b4f080cd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:33.044934 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-02abaf24-492a-427e-93b6-9c2b4f080cd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:33.108815 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02abaf24-492a-427e-93b6-9c2b4f080cd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:33.110236 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02abaf24-492a-427e-93b6-9c2b4f080cd8 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:33.119842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1424/2857] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:19:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:19:36.623120 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa6684-06ca-49b5-96a4-23b5a89df012 service placement] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:36.624594 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-78fa6684-06ca-49b5-96a4-23b5a89df012 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:36.639432 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa6684-06ca-49b5-96a4-23b5a89df012 service placement] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:36.644640 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c4e00-261b-438e-9a28-2ff891f3584a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:36.644828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c4e00-261b-438e-9a28-2ff891f3584a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:36.645105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c4e00-261b-438e-9a28-2ff891f3584a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:36.645376 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c4e00-261b-438e-9a28-2ff891f3584a admin admin] PATH_INFO: `/v3/users/991f6d49ad014318bc9c2f5d57b9deb7` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:36.646537 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c4e00-261b-438e-9a28-2ff891f3584a admin admin] RBAC: Authorizing `identity:delete_user(user_id=991f6d49ad014318bc9c2f5d57b9deb7)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:36.648755 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c4e00-261b-438e-9a28-2ff891f3584a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:36.850617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-006c4e00-261b-438e-9a28-2ff891f3584a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '991f6d49ad014318bc9c2f5d57b9deb7'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:36.863788 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-006c4e00-261b-438e-9a28-2ff891f3584a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '991f6d49ad014318bc9c2f5d57b9deb7'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:36.926635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-006c4e00-261b-438e-9a28-2ff891f3584a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '991f6d49ad014318bc9c2f5d57b9deb7'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:36.934858 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-006c4e00-261b-438e-9a28-2ff891f3584a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '991f6d49ad014318bc9c2f5d57b9deb7'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:36.935557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-006c4e00-261b-438e-9a28-2ff891f3584a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '991f6d49ad014318bc9c2f5d57b9deb7'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:36.941457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1434/2858] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 20:19:36 2022] DELETE /identity/v3/users/991f6d49ad014318bc9c2f5d57b9deb7 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:19:37.186284 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02abaf24-492a-427e-93b6-9c2b4f080cd8 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:37.198803 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02abaf24-492a-427e-93b6-9c2b4f080cd8 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:37.256490 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a6c96-7a5b-405a-b76f-da4be3d601a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:37.257119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a6c96-7a5b-405a-b76f-da4be3d601a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:37.257395 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a6c96-7a5b-405a-b76f-da4be3d601a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:37.258083 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a6c96-7a5b-405a-b76f-da4be3d601a9 admin admin] PATH_INFO: `/v3/projects/dc120bdcb47d4b1b872e2132b8ee9f2e` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:37.259255 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a6c96-7a5b-405a-b76f-da4be3d601a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc120bdcb47d4b1b872e2132b8ee9f2e)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:37.262007 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a6c96-7a5b-405a-b76f-da4be3d601a9 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:37.367611 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-266a6c96-7a5b-405a-b76f-da4be3d601a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc120bdcb47d4b1b872e2132b8ee9f2e', 'request_id': 'req-266a6c96-7a5b-405a-b76f-da4be3d601a9'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:37.370100 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:19:37.374321 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-266a6c96-7a5b-405a-b76f-da4be3d601a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc120bdcb47d4b1b872e2132b8ee9f2e', 'request_id': 'req-266a6c96-7a5b-405a-b76f-da4be3d601a9'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:37.381286 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-266a6c96-7a5b-405a-b76f-da4be3d601a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc120bdcb47d4b1b872e2132b8ee9f2e', 'request_id': 'req-266a6c96-7a5b-405a-b76f-da4be3d601a9'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:37.471549 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-266a6c96-7a5b-405a-b76f-da4be3d601a9 admin admin] The token cache is being invalidate because project dc120bdcb47d4b1b872e2132b8ee9f2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 20:19:37.471842 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-266a6c96-7a5b-405a-b76f-da4be3d601a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:37.472453 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-266a6c96-7a5b-405a-b76f-da4be3d601a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:37.473989 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1425/2859] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 20:19:37 2022] DELETE /identity/v3/projects/dc120bdcb47d4b1b872e2132b8ee9f2e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:19:37.480000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b0386-f819-4f3e-9ccc-39b59eb0e814 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:37.480262 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b0386-f819-4f3e-9ccc-39b59eb0e814 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:37.480534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b0386-f819-4f3e-9ccc-39b59eb0e814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:37.501008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-2f2b0386-f819-4f3e-9ccc-39b59eb0e814 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:19:37.502267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-2f2b0386-f819-4f3e-9ccc-39b59eb0e814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:37.562793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1435/2860] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:19:37.571420 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df574173-8009-4fcb-8d90-51e61c1a6c90 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:37.571726 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df574173-8009-4fcb-8d90-51e61c1a6c90 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:37.571996 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df574173-8009-4fcb-8d90-51e61c1a6c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:37.591802 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-df574173-8009-4fcb-8d90-51e61c1a6c90 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:19:37.593005 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-df574173-8009-4fcb-8d90-51e61c1a6c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:37.600807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1426/2861] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:19:37.606563 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b0386-f819-4f3e-9ccc-39b59eb0e814 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:37.616296 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b0386-f819-4f3e-9ccc-39b59eb0e814 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:37.621580 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ee089-f3ac-462d-a7fe-eaa5bd871f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:37.622124 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ee089-f3ac-462d-a7fe-eaa5bd871f36 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:37.622404 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ee089-f3ac-462d-a7fe-eaa5bd871f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:37.622676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ee089-f3ac-462d-a7fe-eaa5bd871f36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:37.623411 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ee089-f3ac-462d-a7fe-eaa5bd871f36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 20:19:37.624315 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ee089-f3ac-462d-a7fe-eaa5bd871f36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:37.625297 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ee089-f3ac-462d-a7fe-eaa5bd871f36 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:37.637828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1436/2862] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 20:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 20:19:37.642379 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df574173-8009-4fcb-8d90-51e61c1a6c90 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:37.652435 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df574173-8009-4fcb-8d90-51e61c1a6c90 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:37.657272 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12ddd2-36e7-407a-926d-79a992610bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:37.659344 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12ddd2-36e7-407a-926d-79a992610bf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:37.659344 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12ddd2-36e7-407a-926d-79a992610bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:37.659344 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12ddd2-36e7-407a-926d-79a992610bf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:37.659344 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12ddd2-36e7-407a-926d-79a992610bf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:37.661124 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12ddd2-36e7-407a-926d-79a992610bf6 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:37.740619 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1427/2863] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 20:19:37 2022] POST /identity/v3/projects => generated 403 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:19:37.745677 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ee089-f3ac-462d-a7fe-eaa5bd871f36 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:37.756416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ee089-f3ac-462d-a7fe-eaa5bd871f36 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:37.816979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cd93d-8ce0-4307-bf5f-ffc98fe0ced2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:37.817610 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cd93d-8ce0-4307-bf5f-ffc98fe0ced2 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:37.817894 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cd93d-8ce0-4307-bf5f-ffc98fe0ced2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:37.818176 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cd93d-8ce0-4307-bf5f-ffc98fe0ced2 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:37.818832 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cd93d-8ce0-4307-bf5f-ffc98fe0ced2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:37.820791 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cd93d-8ce0-4307-bf5f-ffc98fe0ced2 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:37.829249 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:19:37.929912 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1437/2864] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 20:19:37 2022] POST /identity/v3/users => generated 302 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:19:37.934506 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12ddd2-36e7-407a-926d-79a992610bf6 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:37.944939 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12ddd2-36e7-407a-926d-79a992610bf6 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:37.949946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc442b2-9529-45c4-b8d3-fca77adc90f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:37.955516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc442b2-9529-45c4-b8d3-fca77adc90f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:37.955516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc442b2-9529-45c4-b8d3-fca77adc90f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:37.955516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc442b2-9529-45c4-b8d3-fca77adc90f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:37.955516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc442b2-9529-45c4-b8d3-fca77adc90f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:37.955516 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc442b2-9529-45c4-b8d3-fca77adc90f9 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:37.965242 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1428/2865] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:19:37.969448 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cd93d-8ce0-4307-bf5f-ffc98fe0ced2 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:37.980333 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cd93d-8ce0-4307-bf5f-ffc98fe0ced2 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:37.985087 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8cf2f-05e7-4402-b650-d1a580bddb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:37.985689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8cf2f-05e7-4402-b650-d1a580bddb1e admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:37.985966 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8cf2f-05e7-4402-b650-d1a580bddb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:37.986238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8cf2f-05e7-4402-b650-d1a580bddb1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:37.986904 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8cf2f-05e7-4402-b650-d1a580bddb1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:37.988497 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8cf2f-05e7-4402-b650-d1a580bddb1e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:38.000635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1438/2866] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 20:19:38.004939 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc442b2-9529-45c4-b8d3-fca77adc90f9 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:38.015941 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc442b2-9529-45c4-b8d3-fca77adc90f9 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:38.020624 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56cf5c-48c4-4545-a56f-757ede5edf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:38.021283 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56cf5c-48c4-4545-a56f-757ede5edf91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:38.021564 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56cf5c-48c4-4545-a56f-757ede5edf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:38.021843 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56cf5c-48c4-4545-a56f-757ede5edf91 admin admin] PATH_INFO: `/v3/projects/7964a5056fec4fb28f859ed5ecfd371e/users/8b316b881f17450f89e265aaa9072b1c/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:38.037236 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac56cf5c-48c4-4545-a56f-757ede5edf91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7964a5056fec4fb28f859ed5ecfd371e, user_id=8b316b881f17450f89e265aaa9072b1c, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:38.039926 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac56cf5c-48c4-4545-a56f-757ede5edf91 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:38.107965 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1429/2867] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:19:38 2022] PUT /identity/v3/projects/7964a5056fec4fb28f859ed5ecfd371e/users/8b316b881f17450f89e265aaa9072b1c/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:19:38.113084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31319e3f-b7a5-4ea8-8e35-b7d4be903696 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:38.113372 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31319e3f-b7a5-4ea8-8e35-b7d4be903696 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:38.113637 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31319e3f-b7a5-4ea8-8e35-b7d4be903696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:38.132780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-31319e3f-b7a5-4ea8-8e35-b7d4be903696 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:19:38.133986 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-31319e3f-b7a5-4ea8-8e35-b7d4be903696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:38.195303 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1439/2868] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:19:38.205095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56cf5c-48c4-4545-a56f-757ede5edf91 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:38.206860 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ac56cf5c-48c4-4545-a56f-757ede5edf91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:38.265527 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56cf5c-48c4-4545-a56f-757ede5edf91 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:38.269179 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce2fd2-85f7-4db7-9946-7630d87d128c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:38.269798 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce2fd2-85f7-4db7-9946-7630d87d128c service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:38.270081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce2fd2-85f7-4db7-9946-7630d87d128c service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:38.270357 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce2fd2-85f7-4db7-9946-7630d87d128c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:38.281003 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce2fd2-85f7-4db7-9946-7630d87d128c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:38.282448 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce2fd2-85f7-4db7-9946-7630d87d128c service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:38.293150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1430/2869] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:19:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:19:39.510276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693c753-2659-4e65-90a3-d2cdf971d89e None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:39.511057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693c753-2659-4e65-90a3-d2cdf971d89e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:39.511057 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693c753-2659-4e65-90a3-d2cdf971d89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:39.541587 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-7693c753-2659-4e65-90a3-d2cdf971d89e None None] MFA Rules not processed for user `8b316b881f17450f89e265aaa9072b1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:19:39.543924 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-7693c753-2659-4e65-90a3-d2cdf971d89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:39.612192 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1440/2870] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:19:39 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:19:39.616832 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce2fd2-85f7-4db7-9946-7630d87d128c service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:39.627095 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce2fd2-85f7-4db7-9946-7630d87d128c service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:39.631590 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147bb75-8bea-4e67-bec7-c1f640294ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:39.632188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147bb75-8bea-4e67-bec7-c1f640294ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:39.632463 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147bb75-8bea-4e67-bec7-c1f640294ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:39.632730 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147bb75-8bea-4e67-bec7-c1f640294ba4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:39.633367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147bb75-8bea-4e67-bec7-c1f640294ba4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:39.635503 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147bb75-8bea-4e67-bec7-c1f640294ba4 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:39.745538 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1431/2871] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 20:19:39 2022] POST /identity/v3/projects => generated 405 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:19:39.749528 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693c753-2659-4e65-90a3-d2cdf971d89e None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:39.760531 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693c753-2659-4e65-90a3-d2cdf971d89e None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:39.817430 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b15af-9a00-4799-9809-529a611e9e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:39.818044 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b15af-9a00-4799-9809-529a611e9e62 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:39.818326 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b15af-9a00-4799-9809-529a611e9e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:39.818600 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b15af-9a00-4799-9809-529a611e9e62 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:39.819275 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b15af-9a00-4799-9809-529a611e9e62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:39.821221 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b15af-9a00-4799-9809-529a611e9e62 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:39.893178 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:19:39.899439 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1441/2872] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 20:19:39 2022] POST /identity/v3/users => generated 309 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:19:39.904250 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147bb75-8bea-4e67-bec7-c1f640294ba4 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:39.914654 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147bb75-8bea-4e67-bec7-c1f640294ba4 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:39.919885 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62742689-31ff-4f20-9737-6679efbaaaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:39.920492 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62742689-31ff-4f20-9737-6679efbaaaf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:39.920772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62742689-31ff-4f20-9737-6679efbaaaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:39.921044 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62742689-31ff-4f20-9737-6679efbaaaf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:39.921679 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62742689-31ff-4f20-9737-6679efbaaaf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:39.923329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62742689-31ff-4f20-9737-6679efbaaaf7 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:39.934499 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1432/2873] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:19:39.938740 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b15af-9a00-4799-9809-529a611e9e62 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:39.950259 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b15af-9a00-4799-9809-529a611e9e62 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:39.955107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674fcdd-421f-485f-b1b3-2884c5052d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:39.955759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674fcdd-421f-485f-b1b3-2884c5052d6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:39.956042 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674fcdd-421f-485f-b1b3-2884c5052d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:39.956329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674fcdd-421f-485f-b1b3-2884c5052d6a admin admin] PATH_INFO: `/v3/projects/814b99ee2a5c43b5a254b2517c6077d2/users/9068f569d68445cc8a0ff6bf88c4907b/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:39.973038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674fcdd-421f-485f-b1b3-2884c5052d6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=814b99ee2a5c43b5a254b2517c6077d2, user_id=9068f569d68445cc8a0ff6bf88c4907b, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:39.975744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674fcdd-421f-485f-b1b3-2884c5052d6a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:40.033818 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1442/2874] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:19:39 2022] PUT /identity/v3/projects/814b99ee2a5c43b5a254b2517c6077d2/users/9068f569d68445cc8a0ff6bf88c4907b/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:19:40.037810 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62742689-31ff-4f20-9737-6679efbaaaf7 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:40.049585 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62742689-31ff-4f20-9737-6679efbaaaf7 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:40.095776 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:19:40.108415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3df149-cc64-4c96-b255-ebe9b35bffde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:40.109035 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3df149-cc64-4c96-b255-ebe9b35bffde admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:40.109316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3df149-cc64-4c96-b255-ebe9b35bffde admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:40.109591 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3df149-cc64-4c96-b255-ebe9b35bffde admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:40.110235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3df149-cc64-4c96-b255-ebe9b35bffde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:40.111930 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3df149-cc64-4c96-b255-ebe9b35bffde admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:40.122231 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1433/2875] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:19:40.126571 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674fcdd-421f-485f-b1b3-2884c5052d6a admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:40.137488 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674fcdd-421f-485f-b1b3-2884c5052d6a admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:40.142059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26d188-10f2-4687-915b-d50ff884aabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:40.142714 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26d188-10f2-4687-915b-d50ff884aabd admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:40.143027 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26d188-10f2-4687-915b-d50ff884aabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:40.143298 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26d188-10f2-4687-915b-d50ff884aabd admin admin] PATH_INFO: `/v3/projects/814b99ee2a5c43b5a254b2517c6077d2/users/9068f569d68445cc8a0ff6bf88c4907b/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:40.145187 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26d188-10f2-4687-915b-d50ff884aabd admin admin] RBAC: Authorizing `identity:create_grant(project_id=814b99ee2a5c43b5a254b2517c6077d2, user_id=9068f569d68445cc8a0ff6bf88c4907b, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:40.147846 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26d188-10f2-4687-915b-d50ff884aabd admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:40.159610 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1443/2876] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:19:40 2022] PUT /identity/v3/projects/814b99ee2a5c43b5a254b2517c6077d2/users/9068f569d68445cc8a0ff6bf88c4907b/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:19:41.414483 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20ee12-030f-47dd-8a43-6252d3810f61 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:41.415061 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20ee12-030f-47dd-8a43-6252d3810f61 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:41.415061 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20ee12-030f-47dd-8a43-6252d3810f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:41.445097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-6d20ee12-030f-47dd-8a43-6252d3810f61 None None] MFA Rules not processed for user `9068f569d68445cc8a0ff6bf88c4907b`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:19:41.446331 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-6d20ee12-030f-47dd-8a43-6252d3810f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:41.520582 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1434/2877] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:19:41 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:19:42.821855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26d188-10f2-4687-915b-d50ff884aabd admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:42.906578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26d188-10f2-4687-915b-d50ff884aabd admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:42.911566 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e19dd1-6506-4105-8954-bf21ea4faa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:42.912247 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e19dd1-6506-4105-8954-bf21ea4faa92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:42.912533 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e19dd1-6506-4105-8954-bf21ea4faa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:42.912804 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e19dd1-6506-4105-8954-bf21ea4faa92 admin admin] PATH_INFO: `/v3/users/8b316b881f17450f89e265aaa9072b1c` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:42.914014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e19dd1-6506-4105-8954-bf21ea4faa92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b316b881f17450f89e265aaa9072b1c)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:42.916060 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e19dd1-6506-4105-8954-bf21ea4faa92 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:43.128671 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-74e19dd1-6506-4105-8954-bf21ea4faa92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b316b881f17450f89e265aaa9072b1c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:43.142150 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-74e19dd1-6506-4105-8954-bf21ea4faa92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b316b881f17450f89e265aaa9072b1c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:43.206779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-74e19dd1-6506-4105-8954-bf21ea4faa92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b316b881f17450f89e265aaa9072b1c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:43.215754 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-74e19dd1-6506-4105-8954-bf21ea4faa92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b316b881f17450f89e265aaa9072b1c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:43.216497 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-74e19dd1-6506-4105-8954-bf21ea4faa92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b316b881f17450f89e265aaa9072b1c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:43.223021 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1444/2878] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 20:19:42 2022] DELETE /identity/v3/users/8b316b881f17450f89e265aaa9072b1c => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 20:19:43.476901 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20ee12-030f-47dd-8a43-6252d3810f61 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:43.488773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20ee12-030f-47dd-8a43-6252d3810f61 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:43.555343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:43.555984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:43.556266 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:43.556545 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc admin admin] PATH_INFO: `/v3/projects/7964a5056fec4fb28f859ed5ecfd371e` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:43.557871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7964a5056fec4fb28f859ed5ecfd371e)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:43.559845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:43.677543 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7964a5056fec4fb28f859ed5ecfd371e', 'request_id': 'req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:43.684533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7964a5056fec4fb28f859ed5ecfd371e', 'request_id': 'req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:43.691248 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7964a5056fec4fb28f859ed5ecfd371e', 'request_id': 'req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:43.776612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:19:43.781572 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc admin admin] The token cache is being invalidate because project 7964a5056fec4fb28f859ed5ecfd371e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 20:19:43.781869 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:43.782542 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:43.784296 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1435/2879] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 20:19:43 2022] DELETE /identity/v3/projects/7964a5056fec4fb28f859ed5ecfd371e => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:19:43.788186 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e19dd1-6506-4105-8954-bf21ea4faa92 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:43.796232 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-74e19dd1-6506-4105-8954-bf21ea4faa92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:43.804600 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e19dd1-6506-4105-8954-bf21ea4faa92 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:43.844605 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:19:43.863325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b97fe7-4975-4cd9-80ef-aa81d4e60f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:43.863988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b97fe7-4975-4cd9-80ef-aa81d4e60f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:43.864267 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b97fe7-4975-4cd9-80ef-aa81d4e60f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:43.864538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b97fe7-4975-4cd9-80ef-aa81d4e60f60 admin admin] PATH_INFO: `/v3/users/9068f569d68445cc8a0ff6bf88c4907b` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:43.865679 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b97fe7-4975-4cd9-80ef-aa81d4e60f60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9068f569d68445cc8a0ff6bf88c4907b)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:43.867640 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b97fe7-4975-4cd9-80ef-aa81d4e60f60 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:44.058841 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-41b97fe7-4975-4cd9-80ef-aa81d4e60f60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9068f569d68445cc8a0ff6bf88c4907b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:44.070573 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-41b97fe7-4975-4cd9-80ef-aa81d4e60f60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9068f569d68445cc8a0ff6bf88c4907b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:44.152931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-41b97fe7-4975-4cd9-80ef-aa81d4e60f60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9068f569d68445cc8a0ff6bf88c4907b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:44.160717 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-41b97fe7-4975-4cd9-80ef-aa81d4e60f60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9068f569d68445cc8a0ff6bf88c4907b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:44.161387 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-41b97fe7-4975-4cd9-80ef-aa81d4e60f60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9068f569d68445cc8a0ff6bf88c4907b'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:44.167546 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1445/2880] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 20:19:43 2022] DELETE /identity/v3/users/9068f569d68445cc8a0ff6bf88c4907b => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:19:44.457339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:44.461942 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:44.475078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6294cd-8ecd-47ad-b5f7-baa250c4cbbc admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:44.536373 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f10b51-c85f-48de-8183-f99a8009d696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:44.537115 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f10b51-c85f-48de-8183-f99a8009d696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:44.537273 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f10b51-c85f-48de-8183-f99a8009d696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:44.537548 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f10b51-c85f-48de-8183-f99a8009d696 admin admin] PATH_INFO: `/v3/projects/814b99ee2a5c43b5a254b2517c6077d2` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:44.538672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f10b51-c85f-48de-8183-f99a8009d696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=814b99ee2a5c43b5a254b2517c6077d2)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:44.540609 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f10b51-c85f-48de-8183-f99a8009d696 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:44.679253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-49f10b51-c85f-48de-8183-f99a8009d696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '814b99ee2a5c43b5a254b2517c6077d2', 'request_id': 'req-49f10b51-c85f-48de-8183-f99a8009d696'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:44.685568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-49f10b51-c85f-48de-8183-f99a8009d696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '814b99ee2a5c43b5a254b2517c6077d2', 'request_id': 'req-49f10b51-c85f-48de-8183-f99a8009d696'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:44.691954 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-49f10b51-c85f-48de-8183-f99a8009d696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '814b99ee2a5c43b5a254b2517c6077d2', 'request_id': 'req-49f10b51-c85f-48de-8183-f99a8009d696'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:44.789810 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-49f10b51-c85f-48de-8183-f99a8009d696 admin admin] The token cache is being invalidate because project 814b99ee2a5c43b5a254b2517c6077d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 20:19:44.790026 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-49f10b51-c85f-48de-8183-f99a8009d696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:44.790697 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-49f10b51-c85f-48de-8183-f99a8009d696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:19:44.792786 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1436/2881] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 20:19:44 2022] DELETE /identity/v3/projects/814b99ee2a5c43b5a254b2517c6077d2 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:19:44.798111 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a462ef3-8d20-4543-83c6-89f3d3db366b None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:44.801936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a462ef3-8d20-4543-83c6-89f3d3db366b None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:44.801936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a462ef3-8d20-4543-83c6-89f3d3db366b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:44.820964 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-2a462ef3-8d20-4543-83c6-89f3d3db366b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:19:44.822381 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-2a462ef3-8d20-4543-83c6-89f3d3db366b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:44.888255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1446/2882] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:19:44.896145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383db099-283e-4a67-8c02-46bd00a48437 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:44.896528 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383db099-283e-4a67-8c02-46bd00a48437 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:44.896803 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383db099-283e-4a67-8c02-46bd00a48437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:44.916877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-383db099-283e-4a67-8c02-46bd00a48437 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:19:44.918102 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-383db099-283e-4a67-8c02-46bd00a48437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:44.926533 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1437/2883] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:19:44.930621 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a462ef3-8d20-4543-83c6-89f3d3db366b None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:44.941196 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a462ef3-8d20-4543-83c6-89f3d3db366b None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:44.945767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24083400-c9bc-454f-9c59-605b17245c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:44.946368 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24083400-c9bc-454f-9c59-605b17245c60 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:44.946645 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24083400-c9bc-454f-9c59-605b17245c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:44.946954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24083400-c9bc-454f-9c59-605b17245c60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:44.948265 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24083400-c9bc-454f-9c59-605b17245c60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 20:19:44.948723 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24083400-c9bc-454f-9c59-605b17245c60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:44.950365 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24083400-c9bc-454f-9c59-605b17245c60 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:44.962342 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1447/2884] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 20:19:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 20:19:44.966946 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383db099-283e-4a67-8c02-46bd00a48437 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:44.976400 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383db099-283e-4a67-8c02-46bd00a48437 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:44.981309 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17771fb7-57fd-4f08-8b8b-a582508f64d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:44.981912 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17771fb7-57fd-4f08-8b8b-a582508f64d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:44.982189 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17771fb7-57fd-4f08-8b8b-a582508f64d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:44.982474 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17771fb7-57fd-4f08-8b8b-a582508f64d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:44.983131 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17771fb7-57fd-4f08-8b8b-a582508f64d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:44.985051 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17771fb7-57fd-4f08-8b8b-a582508f64d3 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:45.095116 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1438/2885] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 20:19:44 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 20:19:45.099316 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24083400-c9bc-454f-9c59-605b17245c60 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:45.110897 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24083400-c9bc-454f-9c59-605b17245c60 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:45.157559 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:19:45.171736 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da301af0-0271-40fd-92b4-820aaa55e69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:45.172357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da301af0-0271-40fd-92b4-820aaa55e69a admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:45.172638 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da301af0-0271-40fd-92b4-820aaa55e69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:45.172946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da301af0-0271-40fd-92b4-820aaa55e69a admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:45.173615 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da301af0-0271-40fd-92b4-820aaa55e69a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:45.175697 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da301af0-0271-40fd-92b4-820aaa55e69a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:45.264295 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1448/2886] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 20:19:45 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 20:19:45.268468 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17771fb7-57fd-4f08-8b8b-a582508f64d3 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:45.279884 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17771fb7-57fd-4f08-8b8b-a582508f64d3 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:45.285260 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6627489-872d-442b-8522-83596b048ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:45.285884 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6627489-872d-442b-8522-83596b048ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:45.286321 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6627489-872d-442b-8522-83596b048ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:45.286627 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6627489-872d-442b-8522-83596b048ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:45.287311 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6627489-872d-442b-8522-83596b048ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:45.289109 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6627489-872d-442b-8522-83596b048ee0 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:45.301555 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1439/2887] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:19:45.305689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da301af0-0271-40fd-92b4-820aaa55e69a admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:45.317046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da301af0-0271-40fd-92b4-820aaa55e69a admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:45.321599 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b75f9-fc8b-42d4-9740-8c74787c899f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:45.322201 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b75f9-fc8b-42d4-9740-8c74787c899f admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:45.322480 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b75f9-fc8b-42d4-9740-8c74787c899f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:45.322753 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b75f9-fc8b-42d4-9740-8c74787c899f admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:45.323608 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b75f9-fc8b-42d4-9740-8c74787c899f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:45.325238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b75f9-fc8b-42d4-9740-8c74787c899f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:45.336447 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1449/2888] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:19:45.340616 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6627489-872d-442b-8522-83596b048ee0 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:45.351161 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6627489-872d-442b-8522-83596b048ee0 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:45.356571 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1802f-1e51-4dff-b250-732694a0af65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:45.357293 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1802f-1e51-4dff-b250-732694a0af65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:45.357597 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1802f-1e51-4dff-b250-732694a0af65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:45.357872 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1802f-1e51-4dff-b250-732694a0af65 admin admin] PATH_INFO: `/v3/projects/53b50b009f4e42d79029d0b673350cae/users/3518a7eda9c042da9f7edce46e1b0939/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:45.374917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff1802f-1e51-4dff-b250-732694a0af65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53b50b009f4e42d79029d0b673350cae, user_id=3518a7eda9c042da9f7edce46e1b0939, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:45.377631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff1802f-1e51-4dff-b250-732694a0af65 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:45.465287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1440/2889] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:19:45 2022] PUT /identity/v3/projects/53b50b009f4e42d79029d0b673350cae/users/3518a7eda9c042da9f7edce46e1b0939/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:19:45.470979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfc427-9946-48f3-a852-58ed842ab162 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:45.471274 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfc427-9946-48f3-a852-58ed842ab162 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:45.471552 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfc427-9946-48f3-a852-58ed842ab162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:45.491914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-5dcfc427-9946-48f3-a852-58ed842ab162 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:19:45.493132 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-5dcfc427-9946-48f3-a852-58ed842ab162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:45.559299 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1450/2890] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:19:45.569175 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1802f-1e51-4dff-b250-732694a0af65 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:45.570907 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-aff1802f-1e51-4dff-b250-732694a0af65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:45.599526 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:19:45.630605 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1802f-1e51-4dff-b250-732694a0af65 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:45.634418 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5faad6f-5aac-43f9-b6f3-f124de9e5b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:45.635807 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faad6f-5aac-43f9-b6f3-f124de9e5b36 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:45.636136 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faad6f-5aac-43f9-b6f3-f124de9e5b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:45.636769 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faad6f-5aac-43f9-b6f3-f124de9e5b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:45.646493 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5faad6f-5aac-43f9-b6f3-f124de9e5b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:45.648075 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5faad6f-5aac-43f9-b6f3-f124de9e5b36 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:45.658154 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1441/2891] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:19:46.987220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839410ab-6e22-439b-b809-bc403a911efa None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:46.987761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839410ab-6e22-439b-b809-bc403a911efa None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:46.987761 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839410ab-6e22-439b-b809-bc403a911efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:47.019282 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-839410ab-6e22-439b-b809-bc403a911efa None None] MFA Rules not processed for user `3518a7eda9c042da9f7edce46e1b0939`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:19:47.020371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-839410ab-6e22-439b-b809-bc403a911efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:47.088188 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1451/2892] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:19:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 20:19:47.093059 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5faad6f-5aac-43f9-b6f3-f124de9e5b36 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:47.102779 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5faad6f-5aac-43f9-b6f3-f124de9e5b36 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:47.107743 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39df25-c96b-44dc-9d41-1d2bd6ded622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:47.108359 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39df25-c96b-44dc-9d41-1d2bd6ded622 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:47.108646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39df25-c96b-44dc-9d41-1d2bd6ded622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:47.108922 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39df25-c96b-44dc-9d41-1d2bd6ded622 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:47.110356 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39df25-c96b-44dc-9d41-1d2bd6ded622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:47.112657 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39df25-c96b-44dc-9d41-1d2bd6ded622 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:47.200350 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1442/2893] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 20:19:47 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:19:47.204762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839410ab-6e22-439b-b809-bc403a911efa None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:47.216006 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839410ab-6e22-439b-b809-bc403a911efa None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:47.268033 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:19:47.275379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6f696-7362-475b-97b1-e869d2f3251b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:47.275910 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6f696-7362-475b-97b1-e869d2f3251b admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:47.277028 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6f696-7362-475b-97b1-e869d2f3251b admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:47.277028 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6f696-7362-475b-97b1-e869d2f3251b admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:47.277220 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6f696-7362-475b-97b1-e869d2f3251b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:47.279218 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6f696-7362-475b-97b1-e869d2f3251b admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:47.370861 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1452/2894] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 20:19:47 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:19:47.375078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39df25-c96b-44dc-9d41-1d2bd6ded622 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:47.385622 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39df25-c96b-44dc-9d41-1d2bd6ded622 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:47.390818 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea0294-e1d0-474e-983c-ba9971fab370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:47.391451 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea0294-e1d0-474e-983c-ba9971fab370 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:47.391735 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea0294-e1d0-474e-983c-ba9971fab370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:47.392079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea0294-e1d0-474e-983c-ba9971fab370 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:47.392724 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea0294-e1d0-474e-983c-ba9971fab370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:47.394316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea0294-e1d0-474e-983c-ba9971fab370 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:47.405416 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1443/2895] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:19:47.409827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6f696-7362-475b-97b1-e869d2f3251b admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:47.420751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6f696-7362-475b-97b1-e869d2f3251b admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:47.425533 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367abbc5-9719-4c16-9a5e-e802edb81599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:47.426203 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367abbc5-9719-4c16-9a5e-e802edb81599 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:47.426491 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367abbc5-9719-4c16-9a5e-e802edb81599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:47.426769 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367abbc5-9719-4c16-9a5e-e802edb81599 admin admin] PATH_INFO: `/v3/projects/a38d3ce71172406ea388f76013ff95b1/users/2192e0d24e9947dd8a4f2828f822d852/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:47.443264 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367abbc5-9719-4c16-9a5e-e802edb81599 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a38d3ce71172406ea388f76013ff95b1, user_id=2192e0d24e9947dd8a4f2828f822d852, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:47.445999 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367abbc5-9719-4c16-9a5e-e802edb81599 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:47.522430 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1453/2896] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:19:47 2022] PUT /identity/v3/projects/a38d3ce71172406ea388f76013ff95b1/users/2192e0d24e9947dd8a4f2828f822d852/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:19:47.526849 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea0294-e1d0-474e-983c-ba9971fab370 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:47.537017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea0294-e1d0-474e-983c-ba9971fab370 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:47.597903 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfd451-55c8-4b2f-b18a-dce36238d820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:47.598517 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfd451-55c8-4b2f-b18a-dce36238d820 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:47.598806 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfd451-55c8-4b2f-b18a-dce36238d820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:47.599138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfd451-55c8-4b2f-b18a-dce36238d820 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:47.604325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfd451-55c8-4b2f-b18a-dce36238d820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:47.604325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfd451-55c8-4b2f-b18a-dce36238d820 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:47.611937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1444/2897] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 20:19:47.616470 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367abbc5-9719-4c16-9a5e-e802edb81599 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:47.627746 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367abbc5-9719-4c16-9a5e-e802edb81599 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:47.632365 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81402192-cb3b-46e8-868e-458f54a1d513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:47.633013 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81402192-cb3b-46e8-868e-458f54a1d513 admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:47.633288 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81402192-cb3b-46e8-868e-458f54a1d513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:47.633558 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81402192-cb3b-46e8-868e-458f54a1d513 admin admin] PATH_INFO: `/v3/projects/a38d3ce71172406ea388f76013ff95b1/users/2192e0d24e9947dd8a4f2828f822d852/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:47.635625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81402192-cb3b-46e8-868e-458f54a1d513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a38d3ce71172406ea388f76013ff95b1, user_id=2192e0d24e9947dd8a4f2828f822d852, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:47.638229 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81402192-cb3b-46e8-868e-458f54a1d513 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:47.648435 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1454/2898] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:19:47 2022] PUT /identity/v3/projects/a38d3ce71172406ea388f76013ff95b1/users/2192e0d24e9947dd8a4f2828f822d852/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:19:48.958754 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e9b4c-e667-48a6-8853-95d2250b2a07 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:48.959349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e9b4c-e667-48a6-8853-95d2250b2a07 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:48.959349 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e9b4c-e667-48a6-8853-95d2250b2a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:48.990838 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-050e9b4c-e667-48a6-8853-95d2250b2a07 None None] MFA Rules not processed for user `2192e0d24e9947dd8a4f2828f822d852`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:19:48.992129 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-050e9b4c-e667-48a6-8853-95d2250b2a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:49.071411 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1445/2899] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:19:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:19:49.083641 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8a5d7-015e-412e-882a-89204692bcd3 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:49.084145 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8a5d7-015e-412e-882a-89204692bcd3 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:49.084248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8a5d7-015e-412e-882a-89204692bcd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:49.106226 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-28b8a5d7-015e-412e-882a-89204692bcd3 None None] MFA Rules not processed for user `108a698aa3e6477b9ae896d68f22bd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:19:49.107590 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-28b8a5d7-015e-412e-882a-89204692bcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:49.169148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1455/2900] 10.222.0.59 () {58 vars in 1061 bytes} [Wed Aug 31 20:19:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:19:49.175487 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e9b4c-e667-48a6-8853-95d2250b2a07 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:49.186564 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e9b4c-e667-48a6-8853-95d2250b2a07 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:49.190327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b91c64-61cb-4a92-a514-44e522daf76f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:49.190994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b91c64-61cb-4a92-a514-44e522daf76f service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:49.191281 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b91c64-61cb-4a92-a514-44e522daf76f service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:49.191555 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b91c64-61cb-4a92-a514-44e522daf76f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:49.205854 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:19:49.256868 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b91c64-61cb-4a92-a514-44e522daf76f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:49.258327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b91c64-61cb-4a92-a514-44e522daf76f service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:49.268873 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1446/2901] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:19:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:19:50.942733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8a5d7-015e-412e-882a-89204692bcd3 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:19:50.944216 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-28b8a5d7-015e-412e-882a-89204692bcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:19:50.955600 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8a5d7-015e-412e-882a-89204692bcd3 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:19:50.959277 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943186a0-9b3e-4ea4-b5ae-3bf0514aca9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:19:50.959884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943186a0-9b3e-4ea4-b5ae-3bf0514aca9a service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:19:50.960167 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943186a0-9b3e-4ea4-b5ae-3bf0514aca9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:19:50.960444 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943186a0-9b3e-4ea4-b5ae-3bf0514aca9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:19:50.970616 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943186a0-9b3e-4ea4-b5ae-3bf0514aca9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:19:50.972226 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943186a0-9b3e-4ea4-b5ae-3bf0514aca9a service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:19:50.982676 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1456/2902] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:19:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:19:59.264937 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=21 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:00.977565 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=48,keystone:INSERT=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:09.411138 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b91c64-61cb-4a92-a514-44e522daf76f service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:09.477940 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b91c64-61cb-4a92-a514-44e522daf76f service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:09.482947 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab31d5-589e-4024-9440-608869164554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:09.483580 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab31d5-589e-4024-9440-608869164554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:09.483879 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab31d5-589e-4024-9440-608869164554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:09.484145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab31d5-589e-4024-9440-608869164554 admin admin] PATH_INFO: `/v3/users/3518a7eda9c042da9f7edce46e1b0939` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:09.485328 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab31d5-589e-4024-9440-608869164554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3518a7eda9c042da9f7edce46e1b0939)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:09.487287 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab31d5-589e-4024-9440-608869164554 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:09.700730 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-dfab31d5-589e-4024-9440-608869164554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3518a7eda9c042da9f7edce46e1b0939'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:09.795465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-dfab31d5-589e-4024-9440-608869164554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3518a7eda9c042da9f7edce46e1b0939'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:09.796145 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-dfab31d5-589e-4024-9440-608869164554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3518a7eda9c042da9f7edce46e1b0939'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:09.809253 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-dfab31d5-589e-4024-9440-608869164554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3518a7eda9c042da9f7edce46e1b0939'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:09.809927 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-dfab31d5-589e-4024-9440-608869164554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3518a7eda9c042da9f7edce46e1b0939'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:09.823047 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1447/2903] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 20:20:09 2022] DELETE /identity/v3/users/3518a7eda9c042da9f7edce46e1b0939 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:10.071058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943186a0-9b3e-4ea4-b5ae-3bf0514aca9a service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:10.083775 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943186a0-9b3e-4ea4-b5ae-3bf0514aca9a service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:10.140805 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de9534-ed86-44df-a0a9-e705423a050f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:10.141242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de9534-ed86-44df-a0a9-e705423a050f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:10.141520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de9534-ed86-44df-a0a9-e705423a050f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:10.141813 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de9534-ed86-44df-a0a9-e705423a050f admin admin] PATH_INFO: `/v3/projects/53b50b009f4e42d79029d0b673350cae` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:10.142946 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de9534-ed86-44df-a0a9-e705423a050f admin admin] RBAC: Authorizing `identity:delete_project(project_id=53b50b009f4e42d79029d0b673350cae)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:10.144836 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de9534-ed86-44df-a0a9-e705423a050f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:10.271219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-87de9534-ed86-44df-a0a9-e705423a050f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53b50b009f4e42d79029d0b673350cae', 'request_id': 'req-87de9534-ed86-44df-a0a9-e705423a050f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:10.278323 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-87de9534-ed86-44df-a0a9-e705423a050f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53b50b009f4e42d79029d0b673350cae', 'request_id': 'req-87de9534-ed86-44df-a0a9-e705423a050f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:10.355171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-87de9534-ed86-44df-a0a9-e705423a050f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53b50b009f4e42d79029d0b673350cae', 'request_id': 'req-87de9534-ed86-44df-a0a9-e705423a050f'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:10.367277 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-87de9534-ed86-44df-a0a9-e705423a050f admin admin] The token cache is being invalidate because project 53b50b009f4e42d79029d0b673350cae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 20:20:10.367569 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-87de9534-ed86-44df-a0a9-e705423a050f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:10.368212 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-87de9534-ed86-44df-a0a9-e705423a050f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:10.369694 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1457/2904] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 20:20:10 2022] DELETE /identity/v3/projects/53b50b009f4e42d79029d0b673350cae => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:10.374651 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab31d5-589e-4024-9440-608869164554 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:10.376188 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-dfab31d5-589e-4024-9440-608869164554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:10.390163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab31d5-589e-4024-9440-608869164554 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:10.448557 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e2816-dff1-40e2-90f6-191b9dbaf423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:10.449277 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e2816-dff1-40e2-90f6-191b9dbaf423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:10.449465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e2816-dff1-40e2-90f6-191b9dbaf423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:10.449742 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e2816-dff1-40e2-90f6-191b9dbaf423 admin admin] PATH_INFO: `/v3/users/2192e0d24e9947dd8a4f2828f822d852` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:10.450900 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e2816-dff1-40e2-90f6-191b9dbaf423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2192e0d24e9947dd8a4f2828f822d852)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:10.453038 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e2816-dff1-40e2-90f6-191b9dbaf423 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:10.476137 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:10.698831 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0d7e2816-dff1-40e2-90f6-191b9dbaf423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2192e0d24e9947dd8a4f2828f822d852'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:10.789431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0d7e2816-dff1-40e2-90f6-191b9dbaf423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2192e0d24e9947dd8a4f2828f822d852'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:10.790065 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0d7e2816-dff1-40e2-90f6-191b9dbaf423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2192e0d24e9947dd8a4f2828f822d852'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:10.804273 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0d7e2816-dff1-40e2-90f6-191b9dbaf423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2192e0d24e9947dd8a4f2828f822d852'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:10.804980 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-0d7e2816-dff1-40e2-90f6-191b9dbaf423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2192e0d24e9947dd8a4f2828f822d852'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:10.818121 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1448/2905] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 20:20:10 2022] DELETE /identity/v3/users/2192e0d24e9947dd8a4f2828f822d852 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:11.051128 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de9534-ed86-44df-a0a9-e705423a050f admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:11.052658 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-87de9534-ed86-44df-a0a9-e705423a050f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:11.066533 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de9534-ed86-44df-a0a9-e705423a050f admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:11.129371 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2061eb-807e-4ecb-9a03-14569bc7cc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:11.130039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2061eb-807e-4ecb-9a03-14569bc7cc4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:11.130260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2061eb-807e-4ecb-9a03-14569bc7cc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:11.130539 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2061eb-807e-4ecb-9a03-14569bc7cc4c admin admin] PATH_INFO: `/v3/projects/a38d3ce71172406ea388f76013ff95b1` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:11.131720 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2061eb-807e-4ecb-9a03-14569bc7cc4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a38d3ce71172406ea388f76013ff95b1)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:11.133626 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2061eb-807e-4ecb-9a03-14569bc7cc4c admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:11.255463 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:11.268933 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6a2061eb-807e-4ecb-9a03-14569bc7cc4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a38d3ce71172406ea388f76013ff95b1', 'request_id': 'req-6a2061eb-807e-4ecb-9a03-14569bc7cc4c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:11.274557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6a2061eb-807e-4ecb-9a03-14569bc7cc4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a38d3ce71172406ea388f76013ff95b1', 'request_id': 'req-6a2061eb-807e-4ecb-9a03-14569bc7cc4c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:11.328880 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6a2061eb-807e-4ecb-9a03-14569bc7cc4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a38d3ce71172406ea388f76013ff95b1', 'request_id': 'req-6a2061eb-807e-4ecb-9a03-14569bc7cc4c'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:11.342496 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6a2061eb-807e-4ecb-9a03-14569bc7cc4c admin admin] The token cache is being invalidate because project a38d3ce71172406ea388f76013ff95b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 20:20:11.342909 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6a2061eb-807e-4ecb-9a03-14569bc7cc4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:11.343525 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-6a2061eb-807e-4ecb-9a03-14569bc7cc4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:11.345084 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1458/2906] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 20:20:11 2022] DELETE /identity/v3/projects/a38d3ce71172406ea388f76013ff95b1 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:11.350180 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d8b4c-329b-446e-b6df-350f8f194d58 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:11.350480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d8b4c-329b-446e-b6df-350f8f194d58 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:11.350764 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d8b4c-329b-446e-b6df-350f8f194d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:11.372085 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-1d9d8b4c-329b-446e-b6df-350f8f194d58 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:20:11.373316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-1d9d8b4c-329b-446e-b6df-350f8f194d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:11.434819 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1449/2907] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:20:11.442818 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30dfa7-2385-4ccf-a3b4-ae8a93dbd8a2 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:11.443153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30dfa7-2385-4ccf-a3b4-ae8a93dbd8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:11.443450 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30dfa7-2385-4ccf-a3b4-ae8a93dbd8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:11.464217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-5d30dfa7-2385-4ccf-a3b4-ae8a93dbd8a2 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:20:11.465508 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-5d30dfa7-2385-4ccf-a3b4-ae8a93dbd8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:11.474684 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1459/2908] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:20:11.479065 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d8b4c-329b-446e-b6df-350f8f194d58 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:11.490317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d8b4c-329b-446e-b6df-350f8f194d58 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:11.495107 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874bced-3e71-475b-8112-88baee1202a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:11.495715 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874bced-3e71-475b-8112-88baee1202a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:11.496017 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874bced-3e71-475b-8112-88baee1202a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:11.496292 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874bced-3e71-475b-8112-88baee1202a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:11.496994 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874bced-3e71-475b-8112-88baee1202a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 20:20:11.497302 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874bced-3e71-475b-8112-88baee1202a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:11.499123 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874bced-3e71-475b-8112-88baee1202a7 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:11.514568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1450/2909] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 20:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 20:20:11.517740 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30dfa7-2385-4ccf-a3b4-ae8a93dbd8a2 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:11.527385 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30dfa7-2385-4ccf-a3b4-ae8a93dbd8a2 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:11.532993 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f580ed-da08-4e8c-a9b5-70b31df119ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:11.533607 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f580ed-da08-4e8c-a9b5-70b31df119ba admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:11.533884 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f580ed-da08-4e8c-a9b5-70b31df119ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:11.534155 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f580ed-da08-4e8c-a9b5-70b31df119ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:11.534784 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f580ed-da08-4e8c-a9b5-70b31df119ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:11.536764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f580ed-da08-4e8c-a9b5-70b31df119ba admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:11.672843 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1460/2910] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 20:20:11 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:20:11.677480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874bced-3e71-475b-8112-88baee1202a7 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:11.688530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874bced-3e71-475b-8112-88baee1202a7 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:11.745740 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb420e15-3439-4302-8345-db877719fcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:11.746409 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb420e15-3439-4302-8345-db877719fcb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:11.746688 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb420e15-3439-4302-8345-db877719fcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:11.747001 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb420e15-3439-4302-8345-db877719fcb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:11.747638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb420e15-3439-4302-8345-db877719fcb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:11.750078 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb420e15-3439-4302-8345-db877719fcb8 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:11.832728 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1451/2911] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 20:20:11 2022] POST /identity/v3/users => generated 292 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:20:11.836913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f580ed-da08-4e8c-a9b5-70b31df119ba admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:11.847950 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f580ed-da08-4e8c-a9b5-70b31df119ba admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:11.852652 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366a2e8-4f99-46b8-bf1c-602e41aaf29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:11.853253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366a2e8-4f99-46b8-bf1c-602e41aaf29c admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:11.853543 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366a2e8-4f99-46b8-bf1c-602e41aaf29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:11.853811 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366a2e8-4f99-46b8-bf1c-602e41aaf29c admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:11.854457 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366a2e8-4f99-46b8-bf1c-602e41aaf29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:11.856105 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366a2e8-4f99-46b8-bf1c-602e41aaf29c admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:11.867312 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1461/2912] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:20:11.871404 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb420e15-3439-4302-8345-db877719fcb8 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:11.881991 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb420e15-3439-4302-8345-db877719fcb8 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:11.887049 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0ac99-1569-4317-9627-c4ddd7e6b8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:11.887648 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0ac99-1569-4317-9627-c4ddd7e6b8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:11.887931 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0ac99-1569-4317-9627-c4ddd7e6b8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:11.888205 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0ac99-1569-4317-9627-c4ddd7e6b8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:11.888848 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0ac99-1569-4317-9627-c4ddd7e6b8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:11.891246 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0ac99-1569-4317-9627-c4ddd7e6b8e6 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:11.901984 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1452/2913] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:20:11.906767 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366a2e8-4f99-46b8-bf1c-602e41aaf29c admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:11.916962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366a2e8-4f99-46b8-bf1c-602e41aaf29c admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:11.922035 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbc8a7-82fe-461f-999a-1de0598b321a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:11.922688 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbc8a7-82fe-461f-999a-1de0598b321a admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:11.923011 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbc8a7-82fe-461f-999a-1de0598b321a admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:11.923290 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbc8a7-82fe-461f-999a-1de0598b321a admin admin] PATH_INFO: `/v3/projects/a6b311fd1b6042e4aea7b47876736d4f/users/74e1e79bb20346ef87e65eb322f381e7/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:11.938949 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dbc8a7-82fe-461f-999a-1de0598b321a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6b311fd1b6042e4aea7b47876736d4f, user_id=74e1e79bb20346ef87e65eb322f381e7, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:11.941597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dbc8a7-82fe-461f-999a-1de0598b321a admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:12.020644 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1462/2914] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:20:11 2022] PUT /identity/v3/projects/a6b311fd1b6042e4aea7b47876736d4f/users/74e1e79bb20346ef87e65eb322f381e7/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:12.032062 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635499f4-d82d-4431-a78f-5f696c659288 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:12.035773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635499f4-d82d-4431-a78f-5f696c659288 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:12.035773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635499f4-d82d-4431-a78f-5f696c659288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:12.040757 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:12.061153 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-635499f4-d82d-4431-a78f-5f696c659288 None None] MFA Rules not processed for user `74e1e79bb20346ef87e65eb322f381e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:20:12.063223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-635499f4-d82d-4431-a78f-5f696c659288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:12.128581 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1453/2915] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:20:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:20:12.138332 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbc8a7-82fe-461f-999a-1de0598b321a admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:12.200128 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbc8a7-82fe-461f-999a-1de0598b321a admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:12.204881 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524815e5-9c1d-4ba7-b97a-3ef6aabe3472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:12.205480 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524815e5-9c1d-4ba7-b97a-3ef6aabe3472 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:12.205758 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524815e5-9c1d-4ba7-b97a-3ef6aabe3472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:12.206030 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524815e5-9c1d-4ba7-b97a-3ef6aabe3472 admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:12.206660 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524815e5-9c1d-4ba7-b97a-3ef6aabe3472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:12.208607 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524815e5-9c1d-4ba7-b97a-3ef6aabe3472 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:12.294240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1463/2916] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 20:20:12 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:20:12.298439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635499f4-d82d-4431-a78f-5f696c659288 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:12.309156 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635499f4-d82d-4431-a78f-5f696c659288 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:12.367569 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e46ee4-ec21-43d1-974e-02212324b2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:12.368180 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e46ee4-ec21-43d1-974e-02212324b2a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:12.368465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e46ee4-ec21-43d1-974e-02212324b2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:12.368743 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e46ee4-ec21-43d1-974e-02212324b2a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:12.369436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e46ee4-ec21-43d1-974e-02212324b2a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:12.371382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e46ee4-ec21-43d1-974e-02212324b2a1 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:12.453146 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1454/2917] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 20:20:12 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:20:12.457706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524815e5-9c1d-4ba7-b97a-3ef6aabe3472 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:12.468622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524815e5-9c1d-4ba7-b97a-3ef6aabe3472 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:12.473329 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd859dd2-3908-4601-9010-70bc8b1374ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:12.474756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd859dd2-3908-4601-9010-70bc8b1374ac admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:12.474756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd859dd2-3908-4601-9010-70bc8b1374ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:12.474756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd859dd2-3908-4601-9010-70bc8b1374ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:12.475149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd859dd2-3908-4601-9010-70bc8b1374ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:12.477217 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd859dd2-3908-4601-9010-70bc8b1374ac admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:12.489688 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1464/2918] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 20:20:12.493840 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e46ee4-ec21-43d1-974e-02212324b2a1 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:12.505338 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e46ee4-ec21-43d1-974e-02212324b2a1 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:12.510218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947845db-e40f-4a7c-9313-60a6c6f0b080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:12.511028 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947845db-e40f-4a7c-9313-60a6c6f0b080 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:12.511353 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947845db-e40f-4a7c-9313-60a6c6f0b080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:12.511608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947845db-e40f-4a7c-9313-60a6c6f0b080 admin admin] PATH_INFO: `/v3/projects/e3bd2a2af28a45e0864f17f481073397/users/02857867bd394fd594268ae60e7a0369/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:12.527258 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947845db-e40f-4a7c-9313-60a6c6f0b080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3bd2a2af28a45e0864f17f481073397, user_id=02857867bd394fd594268ae60e7a0369, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:12.529897 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947845db-e40f-4a7c-9313-60a6c6f0b080 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:12.627020 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1455/2919] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:20:12 2022] PUT /identity/v3/projects/e3bd2a2af28a45e0864f17f481073397/users/02857867bd394fd594268ae60e7a0369/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:12.631268 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd859dd2-3908-4601-9010-70bc8b1374ac admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:12.642945 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd859dd2-3908-4601-9010-70bc8b1374ac admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:12.672695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:12.700413 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a510b-9dcb-4259-8171-1933a536f23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:12.701022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a510b-9dcb-4259-8171-1933a536f23f admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:12.701305 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a510b-9dcb-4259-8171-1933a536f23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:12.701578 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a510b-9dcb-4259-8171-1933a536f23f admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:12.702370 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a510b-9dcb-4259-8171-1933a536f23f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:12.704517 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a510b-9dcb-4259-8171-1933a536f23f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:12.714536 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1465/2920] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:20:12.718997 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947845db-e40f-4a7c-9313-60a6c6f0b080 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:12.730166 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947845db-e40f-4a7c-9313-60a6c6f0b080 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:12.735403 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c5fb2-f779-4d15-89a4-b0422f26f3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:12.736055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c5fb2-f779-4d15-89a4-b0422f26f3b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:12.736334 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c5fb2-f779-4d15-89a4-b0422f26f3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:12.736603 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c5fb2-f779-4d15-89a4-b0422f26f3b9 admin admin] PATH_INFO: `/v3/projects/e3bd2a2af28a45e0864f17f481073397/users/02857867bd394fd594268ae60e7a0369/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:12.738517 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c5fb2-f779-4d15-89a4-b0422f26f3b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3bd2a2af28a45e0864f17f481073397, user_id=02857867bd394fd594268ae60e7a0369, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:12.741218 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c5fb2-f779-4d15-89a4-b0422f26f3b9 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:12.752532 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1456/2921] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:20:12 2022] PUT /identity/v3/projects/e3bd2a2af28a45e0864f17f481073397/users/02857867bd394fd594268ae60e7a0369/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 20:20:12.762696 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935a258-cc91-471e-a757-5b82934d72e5 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:12.763272 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935a258-cc91-471e-a757-5b82934d72e5 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:12.763733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935a258-cc91-471e-a757-5b82934d72e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:12.791465 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-9935a258-cc91-471e-a757-5b82934d72e5 None None] MFA Rules not processed for user `02857867bd394fd594268ae60e7a0369`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:20:12.792674 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-9935a258-cc91-471e-a757-5b82934d72e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:12.862733 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1466/2922] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:20:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:20:12.873335 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c5fb2-f779-4d15-89a4-b0422f26f3b9 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:12.875782 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e96c5fb2-f779-4d15-89a4-b0422f26f3b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:12.929537 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:12.935001 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c5fb2-f779-4d15-89a4-b0422f26f3b9 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:12.939672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e5243-9b9d-499a-b689-a8ac620074aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:12.940282 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e5243-9b9d-499a-b689-a8ac620074aa service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:12.940566 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e5243-9b9d-499a-b689-a8ac620074aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:12.940841 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e5243-9b9d-499a-b689-a8ac620074aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:12.997275 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e5243-9b9d-499a-b689-a8ac620074aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:12.998613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e5243-9b9d-499a-b689-a8ac620074aa service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:13.008304 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1457/2923] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:20:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:20:21.694726 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935a258-cc91-471e-a757-5b82934d72e5 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:21.704617 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935a258-cc91-471e-a757-5b82934d72e5 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:21.708895 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8d7b8-c120-4b25-a5d1-2743c8dde61a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:21.709497 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8d7b8-c120-4b25-a5d1-2743c8dde61a service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:21.709779 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8d7b8-c120-4b25-a5d1-2743c8dde61a service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:21.710055 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8d7b8-c120-4b25-a5d1-2743c8dde61a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:21.719720 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8d7b8-c120-4b25-a5d1-2743c8dde61a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:21.721227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8d7b8-c120-4b25-a5d1-2743c8dde61a service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:21.732641 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1467/2924] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:20:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:20:23.004254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=27 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:28.861938 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e5243-9b9d-499a-b689-a8ac620074aa service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:28.926656 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e5243-9b9d-499a-b689-a8ac620074aa service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:28.931511 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67b897-8cf2-47a0-aba7-554de4782ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:28.932151 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67b897-8cf2-47a0-aba7-554de4782ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:28.932439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67b897-8cf2-47a0-aba7-554de4782ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:28.932724 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67b897-8cf2-47a0-aba7-554de4782ba7 admin admin] PATH_INFO: `/v3/users/74e1e79bb20346ef87e65eb322f381e7` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:28.933901 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67b897-8cf2-47a0-aba7-554de4782ba7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74e1e79bb20346ef87e65eb322f381e7)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:28.935887 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67b897-8cf2-47a0-aba7-554de4782ba7 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:29.153018 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-3d67b897-8cf2-47a0-aba7-554de4782ba7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74e1e79bb20346ef87e65eb322f381e7'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:29.234709 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-3d67b897-8cf2-47a0-aba7-554de4782ba7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74e1e79bb20346ef87e65eb322f381e7'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:29.235368 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-3d67b897-8cf2-47a0-aba7-554de4782ba7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74e1e79bb20346ef87e65eb322f381e7'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:29.248517 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-3d67b897-8cf2-47a0-aba7-554de4782ba7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74e1e79bb20346ef87e65eb322f381e7'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:29.249184 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-3d67b897-8cf2-47a0-aba7-554de4782ba7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74e1e79bb20346ef87e65eb322f381e7'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:29.262091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1458/2925] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 20:20:28 2022] DELETE /identity/v3/users/74e1e79bb20346ef87e65eb322f381e7 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:29.266811 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5e45e-f574-4f11-bc36-420f07e389e9 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:29.267357 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5e45e-f574-4f11-bc36-420f07e389e9 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:29.267452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5e45e-f574-4f11-bc36-420f07e389e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:29.289195 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-6bb5e45e-f574-4f11-bc36-420f07e389e9 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:20:29.290452 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6bb5e45e-f574-4f11-bc36-420f07e389e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:29.350606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1468/2926] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:20:29.360642 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67b897-8cf2-47a0-aba7-554de4782ba7 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:29.365343 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-3d67b897-8cf2-47a0-aba7-554de4782ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:29.426164 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67b897-8cf2-47a0-aba7-554de4782ba7 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:29.429773 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e0b10-c9b2-42b7-aecd-97f69ed9da18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:29.431143 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e0b10-c9b2-42b7-aecd-97f69ed9da18 service neutron] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:29.431462 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e0b10-c9b2-42b7-aecd-97f69ed9da18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:29.431791 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e0b10-c9b2-42b7-aecd-97f69ed9da18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:29.440988 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e0b10-c9b2-42b7-aecd-97f69ed9da18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:29.442407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e0b10-c9b2-42b7-aecd-97f69ed9da18 service neutron] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:29.452646 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1459/2927] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:20:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:20:29.896380 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5e45e-f574-4f11-bc36-420f07e389e9 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:29.897970 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-6bb5e45e-f574-4f11-bc36-420f07e389e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:29.911215 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5e45e-f574-4f11-bc36-420f07e389e9 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:29.917146 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5841ba0-8508-49a1-bd05-20c421aed81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:29.918074 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5841ba0-8508-49a1-bd05-20c421aed81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:29.918335 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5841ba0-8508-49a1-bd05-20c421aed81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:29.918612 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5841ba0-8508-49a1-bd05-20c421aed81e admin admin] PATH_INFO: `/v3/projects/a6b311fd1b6042e4aea7b47876736d4f` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:29.919792 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5841ba0-8508-49a1-bd05-20c421aed81e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6b311fd1b6042e4aea7b47876736d4f)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:29.921700 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5841ba0-8508-49a1-bd05-20c421aed81e admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:30.035383 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:30.045416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d5841ba0-8508-49a1-bd05-20c421aed81e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6b311fd1b6042e4aea7b47876736d4f', 'request_id': 'req-d5841ba0-8508-49a1-bd05-20c421aed81e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:30.050954 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d5841ba0-8508-49a1-bd05-20c421aed81e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6b311fd1b6042e4aea7b47876736d4f', 'request_id': 'req-d5841ba0-8508-49a1-bd05-20c421aed81e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:30.160285 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d5841ba0-8508-49a1-bd05-20c421aed81e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6b311fd1b6042e4aea7b47876736d4f', 'request_id': 'req-d5841ba0-8508-49a1-bd05-20c421aed81e'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:30.174692 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d5841ba0-8508-49a1-bd05-20c421aed81e admin admin] The token cache is being invalidate because project a6b311fd1b6042e4aea7b47876736d4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 20:20:30.175043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d5841ba0-8508-49a1-bd05-20c421aed81e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:30.175724 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-d5841ba0-8508-49a1-bd05-20c421aed81e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:30.177253 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1469/2928] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 20:20:29 2022] DELETE /identity/v3/projects/a6b311fd1b6042e4aea7b47876736d4f => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:30.181956 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e0b10-c9b2-42b7-aecd-97f69ed9da18 service neutron] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:30.192482 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e0b10-c9b2-42b7-aecd-97f69ed9da18 service neutron] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:30.212252 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:30.252754 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5a8a9-0cec-435c-8c5e-4a0c21090f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:30.253386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5a8a9-0cec-435c-8c5e-4a0c21090f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:30.253666 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5a8a9-0cec-435c-8c5e-4a0c21090f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:30.253939 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5a8a9-0cec-435c-8c5e-4a0c21090f7e admin admin] PATH_INFO: `/v3/users/02857867bd394fd594268ae60e7a0369` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:30.255105 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5a8a9-0cec-435c-8c5e-4a0c21090f7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=02857867bd394fd594268ae60e7a0369)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:30.257081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5a8a9-0cec-435c-8c5e-4a0c21090f7e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:30.458335 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-59d5a8a9-0cec-435c-8c5e-4a0c21090f7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02857867bd394fd594268ae60e7a0369'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:30.537654 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-59d5a8a9-0cec-435c-8c5e-4a0c21090f7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02857867bd394fd594268ae60e7a0369'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:30.538433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-59d5a8a9-0cec-435c-8c5e-4a0c21090f7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02857867bd394fd594268ae60e7a0369'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:30.553631 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-59d5a8a9-0cec-435c-8c5e-4a0c21090f7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02857867bd394fd594268ae60e7a0369'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:30.554430 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-59d5a8a9-0cec-435c-8c5e-4a0c21090f7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02857867bd394fd594268ae60e7a0369'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:30.568833 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1460/2929] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 20:20:30 2022] DELETE /identity/v3/users/02857867bd394fd594268ae60e7a0369 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:31.025759 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5841ba0-8508-49a1-bd05-20c421aed81e admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:31.027429 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-d5841ba0-8508-49a1-bd05-20c421aed81e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:31.041315 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5841ba0-8508-49a1-bd05-20c421aed81e admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:31.099824 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093291c-84f1-49cb-8443-7f68e441f4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:31.100460 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093291c-84f1-49cb-8443-7f68e441f4eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:31.100741 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093291c-84f1-49cb-8443-7f68e441f4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:31.101014 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093291c-84f1-49cb-8443-7f68e441f4eb admin admin] PATH_INFO: `/v3/projects/e3bd2a2af28a45e0864f17f481073397` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:31.102138 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093291c-84f1-49cb-8443-7f68e441f4eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3bd2a2af28a45e0864f17f481073397)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:31.104793 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093291c-84f1-49cb-8443-7f68e441f4eb admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:31.226541 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-9093291c-84f1-49cb-8443-7f68e441f4eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3bd2a2af28a45e0864f17f481073397', 'request_id': 'req-9093291c-84f1-49cb-8443-7f68e441f4eb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:31.232344 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-9093291c-84f1-49cb-8443-7f68e441f4eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3bd2a2af28a45e0864f17f481073397', 'request_id': 'req-9093291c-84f1-49cb-8443-7f68e441f4eb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:31.311870 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-9093291c-84f1-49cb-8443-7f68e441f4eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3bd2a2af28a45e0864f17f481073397', 'request_id': 'req-9093291c-84f1-49cb-8443-7f68e441f4eb'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:31.324603 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-9093291c-84f1-49cb-8443-7f68e441f4eb admin admin] The token cache is being invalidate because project e3bd2a2af28a45e0864f17f481073397 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81868) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 20:20:31.324899 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-9093291c-84f1-49cb-8443-7f68e441f4eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:31.325557 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-9093291c-84f1-49cb-8443-7f68e441f4eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:31.327117 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1470/2930] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 20:20:31 2022] DELETE /identity/v3/projects/e3bd2a2af28a45e0864f17f481073397 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:31.333259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c6d2a-7480-4077-a397-b9157977096b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:31.333761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c6d2a-7480-4077-a397-b9157977096b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:31.333906 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c6d2a-7480-4077-a397-b9157977096b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:31.354367 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-bc2c6d2a-7480-4077-a397-b9157977096b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:20:31.355614 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-bc2c6d2a-7480-4077-a397-b9157977096b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:31.416976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1461/2931] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:20:31.425292 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9978c6-3b46-4de6-a87a-cc79c13144a0 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:31.425575 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9978c6-3b46-4de6-a87a-cc79c13144a0 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:31.425853 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9978c6-3b46-4de6-a87a-cc79c13144a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:31.446038 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-0f9978c6-3b46-4de6-a87a-cc79c13144a0 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:20:31.448096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-0f9978c6-3b46-4de6-a87a-cc79c13144a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:31.455977 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1471/2932] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:20:31.460629 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c6d2a-7480-4077-a397-b9157977096b None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:31.473186 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c6d2a-7480-4077-a397-b9157977096b None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:31.478276 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2c232-df02-48fb-b063-88d08085201f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:31.478922 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2c232-df02-48fb-b063-88d08085201f admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:31.479205 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2c232-df02-48fb-b063-88d08085201f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:31.479480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2c232-df02-48fb-b063-88d08085201f admin admin] PATH_INFO: `/v3/domains` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:31.480123 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2c232-df02-48fb-b063-88d08085201f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81869) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 20:20:31.480423 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2c232-df02-48fb-b063-88d08085201f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:31.482271 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2c232-df02-48fb-b063-88d08085201f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:31.494862 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1462/2933] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 20:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 20:20:31.499419 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9978c6-3b46-4de6-a87a-cc79c13144a0 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:31.509240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9978c6-3b46-4de6-a87a-cc79c13144a0 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:31.513931 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5edcf62-48ef-4b3d-bbc3-017485fc980c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:31.514561 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edcf62-48ef-4b3d-bbc3-017485fc980c admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:31.514841 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edcf62-48ef-4b3d-bbc3-017485fc980c admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:31.515141 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edcf62-48ef-4b3d-bbc3-017485fc980c admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:31.515763 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5edcf62-48ef-4b3d-bbc3-017485fc980c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:31.517655 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5edcf62-48ef-4b3d-bbc3-017485fc980c admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:31.610485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1472/2934] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 20:20:31 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:20:31.614892 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2c232-df02-48fb-b063-88d08085201f admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:31.625923 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2c232-df02-48fb-b063-88d08085201f admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:31.653655 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:31.686226 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dff3e2-0ccf-454a-935a-7d83a4678d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:31.686844 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dff3e2-0ccf-454a-935a-7d83a4678d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:31.687155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dff3e2-0ccf-454a-935a-7d83a4678d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:31.687432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dff3e2-0ccf-454a-935a-7d83a4678d9b admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:31.688068 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dff3e2-0ccf-454a-935a-7d83a4678d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:31.690045 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dff3e2-0ccf-454a-935a-7d83a4678d9b admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:31.796363 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1463/2935] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 20:20:31 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 20:20:31.800925 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5edcf62-48ef-4b3d-bbc3-017485fc980c admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:31.811832 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5edcf62-48ef-4b3d-bbc3-017485fc980c admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:31.816893 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988540a-6dc1-4717-8c51-dbea7ae9feb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:31.817490 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988540a-6dc1-4717-8c51-dbea7ae9feb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:31.817772 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988540a-6dc1-4717-8c51-dbea7ae9feb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:31.818075 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988540a-6dc1-4717-8c51-dbea7ae9feb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:31.818729 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988540a-6dc1-4717-8c51-dbea7ae9feb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:31.820392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988540a-6dc1-4717-8c51-dbea7ae9feb4 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:31.831455 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1473/2936] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:20:31.835566 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dff3e2-0ccf-454a-935a-7d83a4678d9b admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:31.846531 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dff3e2-0ccf-454a-935a-7d83a4678d9b admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:31.851948 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632348f2-aa54-4df9-b246-371ffcb256c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:31.853197 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632348f2-aa54-4df9-b246-371ffcb256c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:31.853511 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632348f2-aa54-4df9-b246-371ffcb256c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:31.853787 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632348f2-aa54-4df9-b246-371ffcb256c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:31.854442 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632348f2-aa54-4df9-b246-371ffcb256c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:31.856064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632348f2-aa54-4df9-b246-371ffcb256c8 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:31.866962 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1464/2937] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:20:31.871091 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988540a-6dc1-4717-8c51-dbea7ae9feb4 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:31.881039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988540a-6dc1-4717-8c51-dbea7ae9feb4 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:31.885709 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe86ee9-5901-420a-8d06-9718915e71ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:31.886354 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe86ee9-5901-420a-8d06-9718915e71ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:31.886635 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe86ee9-5901-420a-8d06-9718915e71ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:31.886945 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe86ee9-5901-420a-8d06-9718915e71ba admin admin] PATH_INFO: `/v3/projects/802ad27daea042f883a9d6b76b45ebce/users/90e266409e4e4d19bfca2aedfb57b956/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:31.898291 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:31.903330 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe86ee9-5901-420a-8d06-9718915e71ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=802ad27daea042f883a9d6b76b45ebce, user_id=90e266409e4e4d19bfca2aedfb57b956, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:31.906001 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe86ee9-5901-420a-8d06-9718915e71ba admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:32.030138 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1474/2938] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:20:31 2022] PUT /identity/v3/projects/802ad27daea042f883a9d6b76b45ebce/users/90e266409e4e4d19bfca2aedfb57b956/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:32.042443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3923d8-79f5-45a4-a48c-daefe8f95f38 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:32.043032 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3923d8-79f5-45a4-a48c-daefe8f95f38 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:32.043317 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3923d8-79f5-45a4-a48c-daefe8f95f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:32.074076 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-0e3923d8-79f5-45a4-a48c-daefe8f95f38 None None] MFA Rules not processed for user `90e266409e4e4d19bfca2aedfb57b956`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:20:32.075385 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-0e3923d8-79f5-45a4-a48c-daefe8f95f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:32.145745 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1465/2939] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:20:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:20:32.151246 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe86ee9-5901-420a-8d06-9718915e71ba admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:32.213866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe86ee9-5901-420a-8d06-9718915e71ba admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:32.218678 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fbe2a8-0fcd-4ea6-8a4f-c23a0e3cd0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:32.219323 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbe2a8-0fcd-4ea6-8a4f-c23a0e3cd0de admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:32.219608 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbe2a8-0fcd-4ea6-8a4f-c23a0e3cd0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:32.219885 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbe2a8-0fcd-4ea6-8a4f-c23a0e3cd0de admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:32.220534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fbe2a8-0fcd-4ea6-8a4f-c23a0e3cd0de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:32.222443 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fbe2a8-0fcd-4ea6-8a4f-c23a0e3cd0de admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:32.313693 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1475/2940] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 20:20:32 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 20:20:32.317991 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3923d8-79f5-45a4-a48c-daefe8f95f38 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:32.328974 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3923d8-79f5-45a4-a48c-daefe8f95f38 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:32.387505 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e4d1b-3d11-4835-b33d-ad6ce69778ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:32.388222 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e4d1b-3d11-4835-b33d-ad6ce69778ef admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:32.388386 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e4d1b-3d11-4835-b33d-ad6ce69778ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:32.388663 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e4d1b-3d11-4835-b33d-ad6ce69778ef admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:32.389291 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e4d1b-3d11-4835-b33d-ad6ce69778ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:32.391239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e4d1b-3d11-4835-b33d-ad6ce69778ef admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:32.510378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1466/2941] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 20:20:32 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:20:32.514436 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fbe2a8-0fcd-4ea6-8a4f-c23a0e3cd0de admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:32.525919 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fbe2a8-0fcd-4ea6-8a4f-c23a0e3cd0de admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:32.531064 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ada8a-e106-4b96-91b9-4f6a13288fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:32.531780 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ada8a-e106-4b96-91b9-4f6a13288fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:32.532200 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ada8a-e106-4b96-91b9-4f6a13288fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:32.532520 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ada8a-e106-4b96-91b9-4f6a13288fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:32.533209 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ada8a-e106-4b96-91b9-4f6a13288fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:32.535007 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ada8a-e106-4b96-91b9-4f6a13288fd5 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:32.547606 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1476/2942] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:20:32.551706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e4d1b-3d11-4835-b33d-ad6ce69778ef admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:32.562355 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e4d1b-3d11-4835-b33d-ad6ce69778ef admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:32.567424 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef54ccb-c5b0-469b-860a-f2864e2d1852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:32.568023 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef54ccb-c5b0-469b-860a-f2864e2d1852 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:32.568309 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef54ccb-c5b0-469b-860a-f2864e2d1852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:32.568586 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef54ccb-c5b0-469b-860a-f2864e2d1852 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:32.569226 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef54ccb-c5b0-469b-860a-f2864e2d1852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:32.571170 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef54ccb-c5b0-469b-860a-f2864e2d1852 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:32.582345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1467/2943] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:20:32.586601 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ada8a-e106-4b96-91b9-4f6a13288fd5 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:32.597650 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ada8a-e106-4b96-91b9-4f6a13288fd5 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:32.602313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fc6fa-9eda-4801-8f18-679489894dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:32.603757 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fc6fa-9eda-4801-8f18-679489894dde admin admin] REQUEST_METHOD: `PUT` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:32.604039 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fc6fa-9eda-4801-8f18-679489894dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:32.604325 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fc6fa-9eda-4801-8f18-679489894dde admin admin] PATH_INFO: `/v3/projects/d859ea32ef40463e96ec6050918e269b/users/ff6405827aa341ffbff7bb2bbc7f8905/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:32.619813 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9fc6fa-9eda-4801-8f18-679489894dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=d859ea32ef40463e96ec6050918e269b, user_id=ff6405827aa341ffbff7bb2bbc7f8905, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:32.622510 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9fc6fa-9eda-4801-8f18-679489894dde admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:32.738504 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1477/2944] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:20:32 2022] PUT /identity/v3/projects/d859ea32ef40463e96ec6050918e269b/users/ff6405827aa341ffbff7bb2bbc7f8905/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:32.748612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d105755-17e3-4226-9b76-211d5d794be1 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:32.748999 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d105755-17e3-4226-9b76-211d5d794be1 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:32.755208 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d105755-17e3-4226-9b76-211d5d794be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:32.778331 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-7d105755-17e3-4226-9b76-211d5d794be1 None None] MFA Rules not processed for user `ff6405827aa341ffbff7bb2bbc7f8905`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:20:32.779517 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-7d105755-17e3-4226-9b76-211d5d794be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:32.784060 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:32.850737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1468/2945] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:20:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:20:32.855579 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fc6fa-9eda-4801-8f18-679489894dde admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:32.919619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fc6fa-9eda-4801-8f18-679489894dde admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:32.924456 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6d224-e656-4b20-9224-1aee9727062f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:32.925130 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6d224-e656-4b20-9224-1aee9727062f admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:32.925416 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6d224-e656-4b20-9224-1aee9727062f admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:32.925695 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6d224-e656-4b20-9224-1aee9727062f admin admin] PATH_INFO: `/v3/projects` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:32.926341 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6d224-e656-4b20-9224-1aee9727062f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:32.928286 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6d224-e656-4b20-9224-1aee9727062f admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:33.059856 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1478/2946] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 20:20:32 2022] POST /identity/v3/projects => generated 377 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:20:33.063995 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d105755-17e3-4226-9b76-211d5d794be1 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:33.075230 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d105755-17e3-4226-9b76-211d5d794be1 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:33.134141 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f714a-5187-4da1-af19-f08c8607529e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:33.134759 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f714a-5187-4da1-af19-f08c8607529e admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:33.135069 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f714a-5187-4da1-af19-f08c8607529e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:33.135342 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f714a-5187-4da1-af19-f08c8607529e admin admin] PATH_INFO: `/v3/users` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:33.135998 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f714a-5187-4da1-af19-f08c8607529e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:33.137907 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f714a-5187-4da1-af19-f08c8607529e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:33.236439 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1469/2947] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 20:20:33 2022] POST /identity/v3/users => generated 295 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:20:33.240550 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6d224-e656-4b20-9224-1aee9727062f admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:33.251693 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6d224-e656-4b20-9224-1aee9727062f admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:33.256284 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7bbc6-475a-4752-979a-9e921b81c212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:33.256879 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7bbc6-475a-4752-979a-9e921b81c212 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:33.257156 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7bbc6-475a-4752-979a-9e921b81c212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:33.257428 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7bbc6-475a-4752-979a-9e921b81c212 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:33.258099 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7bbc6-475a-4752-979a-9e921b81c212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:33.259847 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7bbc6-475a-4752-979a-9e921b81c212 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:33.271500 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1479/2948] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:20:33.276436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f714a-5187-4da1-af19-f08c8607529e admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:33.287541 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f714a-5187-4da1-af19-f08c8607529e admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:33.293184 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2c2d0-fdc4-4ebb-be6b-b6751db056cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:33.293879 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2c2d0-fdc4-4ebb-be6b-b6751db056cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:33.294164 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2c2d0-fdc4-4ebb-be6b-b6751db056cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:33.294500 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2c2d0-fdc4-4ebb-be6b-b6751db056cb admin admin] PATH_INFO: `/v3/projects/ff5f6551f25446f995673cc995aa2f30/users/48ae1e497c3f48c88da813644d3fccfe/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:33.310894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2c2d0-fdc4-4ebb-be6b-b6751db056cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff5f6551f25446f995673cc995aa2f30, user_id=48ae1e497c3f48c88da813644d3fccfe, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:33.313670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2c2d0-fdc4-4ebb-be6b-b6751db056cb admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:33.378055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1470/2949] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:20:33 2022] PUT /identity/v3/projects/ff5f6551f25446f995673cc995aa2f30/users/48ae1e497c3f48c88da813644d3fccfe/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:33.382046 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7bbc6-475a-4752-979a-9e921b81c212 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:33.392364 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7bbc6-475a-4752-979a-9e921b81c212 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:33.401633 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:33.451777 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873000f-64e1-4239-9f01-dc21bb444111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:33.452390 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873000f-64e1-4239-9f01-dc21bb444111 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:33.452697 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873000f-64e1-4239-9f01-dc21bb444111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:33.452969 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873000f-64e1-4239-9f01-dc21bb444111 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:33.453614 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873000f-64e1-4239-9f01-dc21bb444111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:33.455379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873000f-64e1-4239-9f01-dc21bb444111 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:33.465913 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1480/2950] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:20:33.470090 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2c2d0-fdc4-4ebb-be6b-b6751db056cb admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:33.480715 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2c2d0-fdc4-4ebb-be6b-b6751db056cb admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:33.485327 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcefc7a5-c7bc-44ec-bd2b-c79a158368a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:33.485976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefc7a5-c7bc-44ec-bd2b-c79a158368a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:33.486255 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefc7a5-c7bc-44ec-bd2b-c79a158368a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:33.486530 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefc7a5-c7bc-44ec-bd2b-c79a158368a0 admin admin] PATH_INFO: `/v3/projects/ff5f6551f25446f995673cc995aa2f30/users/48ae1e497c3f48c88da813644d3fccfe/roles/929a670f39fc4a97af9d8ab3f9b5e91a` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:33.488500 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcefc7a5-c7bc-44ec-bd2b-c79a158368a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff5f6551f25446f995673cc995aa2f30, user_id=48ae1e497c3f48c88da813644d3fccfe, role_id=929a670f39fc4a97af9d8ab3f9b5e91a)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:33.491239 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcefc7a5-c7bc-44ec-bd2b-c79a158368a0 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:33.501897 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1471/2951] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:20:33 2022] PUT /identity/v3/projects/ff5f6551f25446f995673cc995aa2f30/users/48ae1e497c3f48c88da813644d3fccfe/roles/929a670f39fc4a97af9d8ab3f9b5e91a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:33.513533 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3745d19-a56c-45e6-904e-e7cba72b120e None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:33.513817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3745d19-a56c-45e6-904e-e7cba72b120e None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:33.514086 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3745d19-a56c-45e6-904e-e7cba72b120e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:33.545135 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-f3745d19-a56c-45e6-904e-e7cba72b120e None None] MFA Rules not processed for user `48ae1e497c3f48c88da813644d3fccfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:20:33.546340 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-f3745d19-a56c-45e6-904e-e7cba72b120e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:33.620510 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1481/2952] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:20:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:20:33.632056 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcefc7a5-c7bc-44ec-bd2b-c79a158368a0 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:33.633820 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-bcefc7a5-c7bc-44ec-bd2b-c79a158368a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:33.692696 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcefc7a5-c7bc-44ec-bd2b-c79a158368a0 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:33.696389 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346cbc5-c739-447e-a3ff-94fd20bb6740 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:33.697608 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346cbc5-c739-447e-a3ff-94fd20bb6740 service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:33.697870 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346cbc5-c739-447e-a3ff-94fd20bb6740 service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:33.698204 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346cbc5-c739-447e-a3ff-94fd20bb6740 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:33.707899 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:33.755913 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346cbc5-c739-447e-a3ff-94fd20bb6740 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:33.757436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346cbc5-c739-447e-a3ff-94fd20bb6740 service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:33.767072 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1472/2953] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:20:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:20:37.084794 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3745d19-a56c-45e6-904e-e7cba72b120e None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:37.094849 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3745d19-a56c-45e6-904e-e7cba72b120e None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:37.099948 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635f083-f7be-4f93-9b13-37aa492cbd61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:37.100553 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635f083-f7be-4f93-9b13-37aa492cbd61 service cinder] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:37.100835 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635f083-f7be-4f93-9b13-37aa492cbd61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:37.101112 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635f083-f7be-4f93-9b13-37aa492cbd61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:37.157694 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635f083-f7be-4f93-9b13-37aa492cbd61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:37.159191 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635f083-f7be-4f93-9b13-37aa492cbd61 service cinder] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:37.168857 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1482/2954] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:20:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:20:38.862683 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346cbc5-c739-447e-a3ff-94fd20bb6740 service cinder] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:38.871207 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346cbc5-c739-447e-a3ff-94fd20bb6740 service cinder] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:38.875400 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7aef4-dfb2-439d-999a-98d3702507ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:38.876107 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7aef4-dfb2-439d-999a-98d3702507ed service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:38.876392 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7aef4-dfb2-439d-999a-98d3702507ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:38.876687 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7aef4-dfb2-439d-999a-98d3702507ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:38.885897 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7aef4-dfb2-439d-999a-98d3702507ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:38.887525 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7aef4-dfb2-439d-999a-98d3702507ed service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:38.896877 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1473/2955] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:20:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:20:44.975355 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635f083-f7be-4f93-9b13-37aa492cbd61 service cinder] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:45.009249 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=100 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:45.036855 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635f083-f7be-4f93-9b13-37aa492cbd61 service cinder] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:45.041756 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355fb78-89b5-415e-ad9d-98dd20d162f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:45.042622 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355fb78-89b5-415e-ad9d-98dd20d162f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:45.042928 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355fb78-89b5-415e-ad9d-98dd20d162f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:45.043205 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355fb78-89b5-415e-ad9d-98dd20d162f2 admin admin] PATH_INFO: `/v3/users/90e266409e4e4d19bfca2aedfb57b956` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:45.044313 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355fb78-89b5-415e-ad9d-98dd20d162f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90e266409e4e4d19bfca2aedfb57b956)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:45.046964 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355fb78-89b5-415e-ad9d-98dd20d162f2 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:45.248979 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2355fb78-89b5-415e-ad9d-98dd20d162f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90e266409e4e4d19bfca2aedfb57b956'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:45.260790 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2355fb78-89b5-415e-ad9d-98dd20d162f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90e266409e4e4d19bfca2aedfb57b956'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:45.345459 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2355fb78-89b5-415e-ad9d-98dd20d162f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90e266409e4e4d19bfca2aedfb57b956'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:45.353485 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2355fb78-89b5-415e-ad9d-98dd20d162f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90e266409e4e4d19bfca2aedfb57b956'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:45.354171 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-2355fb78-89b5-415e-ad9d-98dd20d162f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90e266409e4e4d19bfca2aedfb57b956'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:45.361073 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1483/2956] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 20:20:44 2022] DELETE /identity/v3/users/90e266409e4e4d19bfca2aedfb57b956 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:45.365589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad903450-81db-4d8e-baac-cd9c5a7d290b None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:45.366055 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad903450-81db-4d8e-baac-cd9c5a7d290b None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:45.366185 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad903450-81db-4d8e-baac-cd9c5a7d290b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:45.386085 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-ad903450-81db-4d8e-baac-cd9c5a7d290b None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:20:45.387384 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ad903450-81db-4d8e-baac-cd9c5a7d290b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:45.447436 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1474/2957] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:20:45.457920 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355fb78-89b5-415e-ad9d-98dd20d162f2 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:45.459379 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-2355fb78-89b5-415e-ad9d-98dd20d162f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:45.516908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355fb78-89b5-415e-ad9d-98dd20d162f2 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:45.520404 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff10eba-f5de-4d5a-8da6-8dedb91b7777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:45.521043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff10eba-f5de-4d5a-8da6-8dedb91b7777 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:45.521321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff10eba-f5de-4d5a-8da6-8dedb91b7777 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:45.521594 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff10eba-f5de-4d5a-8da6-8dedb91b7777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:45.531289 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff10eba-f5de-4d5a-8da6-8dedb91b7777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:45.532742 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff10eba-f5de-4d5a-8da6-8dedb91b7777 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:45.542619 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1484/2958] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:20:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:20:45.993151 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad903450-81db-4d8e-baac-cd9c5a7d290b None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:45.995237 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-ad903450-81db-4d8e-baac-cd9c5a7d290b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:46.008793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad903450-81db-4d8e-baac-cd9c5a7d290b None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:46.014155 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e7959-ec7a-4890-9e5a-738b1c059d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:46.014800 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e7959-ec7a-4890-9e5a-738b1c059d2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:46.015117 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e7959-ec7a-4890-9e5a-738b1c059d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:46.015395 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e7959-ec7a-4890-9e5a-738b1c059d2d admin admin] PATH_INFO: `/v3/projects/802ad27daea042f883a9d6b76b45ebce` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:46.016524 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e7959-ec7a-4890-9e5a-738b1c059d2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=802ad27daea042f883a9d6b76b45ebce)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:46.018470 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e7959-ec7a-4890-9e5a-738b1c059d2d admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:46.225097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-911e7959-ec7a-4890-9e5a-738b1c059d2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '802ad27daea042f883a9d6b76b45ebce', 'request_id': 'req-911e7959-ec7a-4890-9e5a-738b1c059d2d'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:46.230710 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-911e7959-ec7a-4890-9e5a-738b1c059d2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '802ad27daea042f883a9d6b76b45ebce', 'request_id': 'req-911e7959-ec7a-4890-9e5a-738b1c059d2d'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:46.241431 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-911e7959-ec7a-4890-9e5a-738b1c059d2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '802ad27daea042f883a9d6b76b45ebce', 'request_id': 'req-911e7959-ec7a-4890-9e5a-738b1c059d2d'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:46.377777 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-911e7959-ec7a-4890-9e5a-738b1c059d2d admin admin] The token cache is being invalidate because project 802ad27daea042f883a9d6b76b45ebce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 20:20:46.378456 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-911e7959-ec7a-4890-9e5a-738b1c059d2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:46.378676 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-911e7959-ec7a-4890-9e5a-738b1c059d2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:46.380279 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1475/2959] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 20:20:45 2022] DELETE /identity/v3/projects/802ad27daea042f883a9d6b76b45ebce => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:46.384269 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff10eba-f5de-4d5a-8da6-8dedb91b7777 service neutron] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:46.394102 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff10eba-f5de-4d5a-8da6-8dedb91b7777 service neutron] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:46.444871 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:46.451817 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29d7fb-d6ba-4f24-8404-dcd5974e6404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:46.452473 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29d7fb-d6ba-4f24-8404-dcd5974e6404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:46.452751 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29d7fb-d6ba-4f24-8404-dcd5974e6404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:46.453022 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29d7fb-d6ba-4f24-8404-dcd5974e6404 admin admin] PATH_INFO: `/v3/users/ff6405827aa341ffbff7bb2bbc7f8905` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:46.454153 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed29d7fb-d6ba-4f24-8404-dcd5974e6404 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff6405827aa341ffbff7bb2bbc7f8905)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:46.456115 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed29d7fb-d6ba-4f24-8404-dcd5974e6404 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:46.693980 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ed29d7fb-d6ba-4f24-8404-dcd5974e6404 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff6405827aa341ffbff7bb2bbc7f8905'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:46.705605 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ed29d7fb-d6ba-4f24-8404-dcd5974e6404 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff6405827aa341ffbff7bb2bbc7f8905'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:46.788691 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ed29d7fb-d6ba-4f24-8404-dcd5974e6404 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff6405827aa341ffbff7bb2bbc7f8905'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:46.796674 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ed29d7fb-d6ba-4f24-8404-dcd5974e6404 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff6405827aa341ffbff7bb2bbc7f8905'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:46.797501 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-ed29d7fb-d6ba-4f24-8404-dcd5974e6404 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff6405827aa341ffbff7bb2bbc7f8905'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:46.803687 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1485/2960] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 20:20:46 2022] DELETE /identity/v3/users/ff6405827aa341ffbff7bb2bbc7f8905 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:47.202589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e7959-ec7a-4890-9e5a-738b1c059d2d admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:47.205111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-911e7959-ec7a-4890-9e5a-738b1c059d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:47.218480 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e7959-ec7a-4890-9e5a-738b1c059d2d admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:47.240010 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:47.282041 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e2a36-f80b-4cc7-8f3f-64261031fd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:47.282723 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e2a36-f80b-4cc7-8f3f-64261031fd45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:47.283038 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e2a36-f80b-4cc7-8f3f-64261031fd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:47.283316 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e2a36-f80b-4cc7-8f3f-64261031fd45 admin admin] PATH_INFO: `/v3/projects/d859ea32ef40463e96ec6050918e269b` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:47.284464 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e2a36-f80b-4cc7-8f3f-64261031fd45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d859ea32ef40463e96ec6050918e269b)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:47.286564 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e2a36-f80b-4cc7-8f3f-64261031fd45 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:47.419220 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a53e2a36-f80b-4cc7-8f3f-64261031fd45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd859ea32ef40463e96ec6050918e269b', 'request_id': 'req-a53e2a36-f80b-4cc7-8f3f-64261031fd45'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:47.425235 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a53e2a36-f80b-4cc7-8f3f-64261031fd45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd859ea32ef40463e96ec6050918e269b', 'request_id': 'req-a53e2a36-f80b-4cc7-8f3f-64261031fd45'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:47.432223 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a53e2a36-f80b-4cc7-8f3f-64261031fd45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd859ea32ef40463e96ec6050918e269b', 'request_id': 'req-a53e2a36-f80b-4cc7-8f3f-64261031fd45'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:47.504589 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a53e2a36-f80b-4cc7-8f3f-64261031fd45 admin admin] The token cache is being invalidate because project d859ea32ef40463e96ec6050918e269b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 20:20:47.504793 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a53e2a36-f80b-4cc7-8f3f-64261031fd45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:47.505438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-a53e2a36-f80b-4cc7-8f3f-64261031fd45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:47.507012 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1476/2961] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 20:20:47 2022] DELETE /identity/v3/projects/d859ea32ef40463e96ec6050918e269b => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 20:20:47.510994 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29d7fb-d6ba-4f24-8404-dcd5974e6404 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:47.512423 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-ed29d7fb-d6ba-4f24-8404-dcd5974e6404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:47.526634 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29d7fb-d6ba-4f24-8404-dcd5974e6404 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:47.583501 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4497fa0-c4fb-45e8-88e1-b7904cfb0989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:47.584138 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4497fa0-c4fb-45e8-88e1-b7904cfb0989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:47.589701 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4497fa0-c4fb-45e8-88e1-b7904cfb0989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:47.589701 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4497fa0-c4fb-45e8-88e1-b7904cfb0989 admin admin] PATH_INFO: `/v3/users/48ae1e497c3f48c88da813644d3fccfe` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:47.589701 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4497fa0-c4fb-45e8-88e1-b7904cfb0989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48ae1e497c3f48c88da813644d3fccfe)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:47.589701 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4497fa0-c4fb-45e8-88e1-b7904cfb0989 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:47.759420 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e4497fa0-c4fb-45e8-88e1-b7904cfb0989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48ae1e497c3f48c88da813644d3fccfe'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:47.773241 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e4497fa0-c4fb-45e8-88e1-b7904cfb0989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48ae1e497c3f48c88da813644d3fccfe'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:47.863095 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e4497fa0-c4fb-45e8-88e1-b7904cfb0989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48ae1e497c3f48c88da813644d3fccfe'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:47.868262 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:47.871588 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e4497fa0-c4fb-45e8-88e1-b7904cfb0989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48ae1e497c3f48c88da813644d3fccfe'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:47.872265 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-e4497fa0-c4fb-45e8-88e1-b7904cfb0989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48ae1e497c3f48c88da813644d3fccfe'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:47.878693 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1486/2962] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 20:20:47 2022] DELETE /identity/v3/users/48ae1e497c3f48c88da813644d3fccfe => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:48.334034 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e2a36-f80b-4cc7-8f3f-64261031fd45 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:48.335726 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-a53e2a36-f80b-4cc7-8f3f-64261031fd45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:48.349303 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e2a36-f80b-4cc7-8f3f-64261031fd45 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:48.409150 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f75ff-0be0-44e7-983f-67361ec3c65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:48.409782 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f75ff-0be0-44e7-983f-67361ec3c65c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:48.410064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f75ff-0be0-44e7-983f-67361ec3c65c admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:48.410388 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f75ff-0be0-44e7-983f-67361ec3c65c admin admin] PATH_INFO: `/v3/projects/ff5f6551f25446f995673cc995aa2f30` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:48.411535 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f75ff-0be0-44e7-983f-67361ec3c65c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff5f6551f25446f995673cc995aa2f30)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:48.413443 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f75ff-0be0-44e7-983f-67361ec3c65c admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:48.569404 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c52f75ff-0be0-44e7-983f-67361ec3c65c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff5f6551f25446f995673cc995aa2f30', 'request_id': 'req-c52f75ff-0be0-44e7-983f-67361ec3c65c'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:48.573438 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:48.575995 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c52f75ff-0be0-44e7-983f-67361ec3c65c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff5f6551f25446f995673cc995aa2f30', 'request_id': 'req-c52f75ff-0be0-44e7-983f-67361ec3c65c'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:48.583276 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c52f75ff-0be0-44e7-983f-67361ec3c65c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff5f6551f25446f995673cc995aa2f30', 'request_id': 'req-c52f75ff-0be0-44e7-983f-67361ec3c65c'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:48.734754 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c52f75ff-0be0-44e7-983f-67361ec3c65c admin admin] The token cache is being invalidate because project ff5f6551f25446f995673cc995aa2f30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 20:20:48.735079 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c52f75ff-0be0-44e7-983f-67361ec3c65c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:48.735794 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-c52f75ff-0be0-44e7-983f-67361ec3c65c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:20:48.737414 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1477/2963] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 20:20:48 2022] DELETE /identity/v3/projects/ff5f6551f25446f995673cc995aa2f30 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:48.743712 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87020828-df1d-483e-ab6a-c4eef9bb2493 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:48.744276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87020828-df1d-483e-ab6a-c4eef9bb2493 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:48.744276 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87020828-df1d-483e-ab6a-c4eef9bb2493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:48.766403 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-87020828-df1d-483e-ab6a-c4eef9bb2493 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:20:48.767625 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-87020828-df1d-483e-ab6a-c4eef9bb2493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:48.826226 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1487/2964] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:20:48.834789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd9b09-b0ac-4661-be8d-4bdfddf6f09d None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:48.835064 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd9b09-b0ac-4661-be8d-4bdfddf6f09d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:48.835346 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd9b09-b0ac-4661-be8d-4bdfddf6f09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:48.856665 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-e8bd9b09-b0ac-4661-be8d-4bdfddf6f09d None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:20:48.857871 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-e8bd9b09-b0ac-4661-be8d-4bdfddf6f09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:48.865925 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1478/2965] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:20:48.870466 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87020828-df1d-483e-ab6a-c4eef9bb2493 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:48.881007 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87020828-df1d-483e-ab6a-c4eef9bb2493 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:48.885679 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf5398-0bb1-445c-8330-17510e70bb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:48.886283 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf5398-0bb1-445c-8330-17510e70bb1d admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:48.886561 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf5398-0bb1-445c-8330-17510e70bb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:48.886834 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf5398-0bb1-445c-8330-17510e70bb1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:48.887522 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf5398-0bb1-445c-8330-17510e70bb1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 20:20:48.887828 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf5398-0bb1-445c-8330-17510e70bb1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:48.889554 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf5398-0bb1-445c-8330-17510e70bb1d admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:48.901921 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1488/2966] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 20:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 20:20:48.906350 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd9b09-b0ac-4661-be8d-4bdfddf6f09d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:48.915561 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd9b09-b0ac-4661-be8d-4bdfddf6f09d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:48.920453 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986b629-4086-44a8-aa86-45d67488f024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:48.921379 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986b629-4086-44a8-aa86-45d67488f024 admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:48.921527 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986b629-4086-44a8-aa86-45d67488f024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:48.921710 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986b629-4086-44a8-aa86-45d67488f024 admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:48.922467 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986b629-4086-44a8-aa86-45d67488f024 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:48.924366 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986b629-4086-44a8-aa86-45d67488f024 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:49.026952 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1479/2967] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 20:20:48 2022] POST /identity/v3/projects => generated 391 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 20:20:49.031551 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf5398-0bb1-445c-8330-17510e70bb1d admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:49.042186 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf5398-0bb1-445c-8330-17510e70bb1d admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:49.100161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd34a4a-d1fb-4bbd-a886-f16e95daaff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:49.100682 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd34a4a-d1fb-4bbd-a886-f16e95daaff3 admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:49.100965 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd34a4a-d1fb-4bbd-a886-f16e95daaff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:49.101242 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd34a4a-d1fb-4bbd-a886-f16e95daaff3 admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:49.101930 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd34a4a-d1fb-4bbd-a886-f16e95daaff3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:49.103936 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd34a4a-d1fb-4bbd-a886-f16e95daaff3 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:49.204058 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1489/2968] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 20:20:49 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 20:20:49.208259 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986b629-4086-44a8-aa86-45d67488f024 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:49.219243 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986b629-4086-44a8-aa86-45d67488f024 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:49.224276 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe31b3-c98c-4500-b22b-4b24b64f82dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:49.224910 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe31b3-c98c-4500-b22b-4b24b64f82dc admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:49.225185 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe31b3-c98c-4500-b22b-4b24b64f82dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:49.225455 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe31b3-c98c-4500-b22b-4b24b64f82dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:49.226098 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe31b3-c98c-4500-b22b-4b24b64f82dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:49.227853 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe31b3-c98c-4500-b22b-4b24b64f82dc admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:49.247397 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1480/2969] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:20:49.252255 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd34a4a-d1fb-4bbd-a886-f16e95daaff3 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:49.267115 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd34a4a-d1fb-4bbd-a886-f16e95daaff3 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:49.272234 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bbf7a-89ce-4982-88a7-ffe3b6bd66a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:49.272866 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bbf7a-89ce-4982-88a7-ffe3b6bd66a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:49.273142 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bbf7a-89ce-4982-88a7-ffe3b6bd66a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:49.273587 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bbf7a-89ce-4982-88a7-ffe3b6bd66a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:49.274148 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999bbf7a-89ce-4982-88a7-ffe3b6bd66a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:49.275802 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999bbf7a-89ce-4982-88a7-ffe3b6bd66a2 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:49.295059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1490/2970] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:20:49.300294 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe31b3-c98c-4500-b22b-4b24b64f82dc admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:49.311027 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe31b3-c98c-4500-b22b-4b24b64f82dc admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:49.317938 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78225985-5d38-40d6-a82f-ebbf3b44bbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:49.318917 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78225985-5d38-40d6-a82f-ebbf3b44bbca admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:49.319254 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78225985-5d38-40d6-a82f-ebbf3b44bbca admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:49.319540 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78225985-5d38-40d6-a82f-ebbf3b44bbca admin admin] PATH_INFO: `/v3/projects/4c84aa2376c247e197daa4eaf6a093ab/users/22393858a01c41ac8b3fc64de13a29f7/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:49.336976 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78225985-5d38-40d6-a82f-ebbf3b44bbca admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c84aa2376c247e197daa4eaf6a093ab, user_id=22393858a01c41ac8b3fc64de13a29f7, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:49.339716 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78225985-5d38-40d6-a82f-ebbf3b44bbca admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:49.437876 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1481/2971] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:20:49 2022] PUT /identity/v3/projects/4c84aa2376c247e197daa4eaf6a093ab/users/22393858a01c41ac8b3fc64de13a29f7/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:20:49.448392 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa1d18-8dce-4611-a022-ca1df0883023 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:49.448689 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa1d18-8dce-4611-a022-ca1df0883023 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:49.448988 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa1d18-8dce-4611-a022-ca1df0883023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:49.478983 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-57aa1d18-8dce-4611-a022-ca1df0883023 None None] MFA Rules not processed for user `22393858a01c41ac8b3fc64de13a29f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:20:49.480238 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-57aa1d18-8dce-4611-a022-ca1df0883023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:49.536953 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:49.557102 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1491/2972] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:20:49 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:20:49.568182 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78225985-5d38-40d6-a82f-ebbf3b44bbca admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:20:49.570613 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-78225985-5d38-40d6-a82f-ebbf3b44bbca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:20:49.638162 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78225985-5d38-40d6-a82f-ebbf3b44bbca admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:20:49.642203 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7756dc2-7d49-415c-9193-54056311ea2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:20:49.642821 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7756dc2-7d49-415c-9193-54056311ea2e service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:20:49.643133 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7756dc2-7d49-415c-9193-54056311ea2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:20:49.643407 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7756dc2-7d49-415c-9193-54056311ea2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:20:49.652415 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7756dc2-7d49-415c-9193-54056311ea2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:20:49.653987 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7756dc2-7d49-415c-9193-54056311ea2e service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:20:49.664365 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1482/2973] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:20:49 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:20:59.617555 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=5 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:20:59.659789 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:DELETE=2,keystone:SELECT=58,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:21:13.793713 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa1d18-8dce-4611-a022-ca1df0883023 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:21:13.854592 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa1d18-8dce-4611-a022-ca1df0883023 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:21:13.859059 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708c2e9-f236-4c33-ba06-b7b58ab0de72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:21:13.859680 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708c2e9-f236-4c33-ba06-b7b58ab0de72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:13.859957 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708c2e9-f236-4c33-ba06-b7b58ab0de72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:13.860227 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708c2e9-f236-4c33-ba06-b7b58ab0de72 admin admin] PATH_INFO: `/v3/users/22393858a01c41ac8b3fc64de13a29f7` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:13.861352 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708c2e9-f236-4c33-ba06-b7b58ab0de72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22393858a01c41ac8b3fc64de13a29f7)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:21:13.863281 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708c2e9-f236-4c33-ba06-b7b58ab0de72 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:21:14.082207 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3708c2e9-f236-4c33-ba06-b7b58ab0de72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22393858a01c41ac8b3fc64de13a29f7'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:14.095086 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3708c2e9-f236-4c33-ba06-b7b58ab0de72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22393858a01c41ac8b3fc64de13a29f7'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:14.162083 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3708c2e9-f236-4c33-ba06-b7b58ab0de72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22393858a01c41ac8b3fc64de13a29f7'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:14.170032 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3708c2e9-f236-4c33-ba06-b7b58ab0de72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22393858a01c41ac8b3fc64de13a29f7'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:14.170731 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-3708c2e9-f236-4c33-ba06-b7b58ab0de72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22393858a01c41ac8b3fc64de13a29f7'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:14.178149 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1492/2974] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 20:21:13 2022] DELETE /identity/v3/users/22393858a01c41ac8b3fc64de13a29f7 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:21:14.182772 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b104bb-2daa-4c38-ac4a-64751616274d None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:14.184960 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b104bb-2daa-4c38-ac4a-64751616274d None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:14.184960 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b104bb-2daa-4c38-ac4a-64751616274d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:14.203378 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-b9b104bb-2daa-4c38-ac4a-64751616274d None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:21:14.204673 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b9b104bb-2daa-4c38-ac4a-64751616274d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:21:14.270163 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1483/2975] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:21:14.280054 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708c2e9-f236-4c33-ba06-b7b58ab0de72 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:21:14.281620 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-3708c2e9-f236-4c33-ba06-b7b58ab0de72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:21:14.343489 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708c2e9-f236-4c33-ba06-b7b58ab0de72 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:21:14.347000 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbd29e-575b-4451-b7f1-aa07af156970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:21:14.347600 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbd29e-575b-4451-b7f1-aa07af156970 service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:14.347879 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbd29e-575b-4451-b7f1-aa07af156970 service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:14.348151 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbd29e-575b-4451-b7f1-aa07af156970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:14.358019 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbbd29e-575b-4451-b7f1-aa07af156970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:21:14.359487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbbd29e-575b-4451-b7f1-aa07af156970 service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:21:14.368908 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1493/2976] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:21:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:21:14.804014 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b104bb-2daa-4c38-ac4a-64751616274d None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:21:14.805433 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-b9b104bb-2daa-4c38-ac4a-64751616274d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:21:14.817718 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b104bb-2daa-4c38-ac4a-64751616274d None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:21:14.823250 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e32ace-b707-4fd9-8d47-8b60564e9a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:21:14.824022 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e32ace-b707-4fd9-8d47-8b60564e9a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:14.824301 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e32ace-b707-4fd9-8d47-8b60564e9a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:14.824574 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e32ace-b707-4fd9-8d47-8b60564e9a7e admin admin] PATH_INFO: `/v3/projects/4c84aa2376c247e197daa4eaf6a093ab` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:14.825697 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e32ace-b707-4fd9-8d47-8b60564e9a7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c84aa2376c247e197daa4eaf6a093ab)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:21:14.827626 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e32ace-b707-4fd9-8d47-8b60564e9a7e admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:21:14.957320 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-51e32ace-b707-4fd9-8d47-8b60564e9a7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c84aa2376c247e197daa4eaf6a093ab', 'request_id': 'req-51e32ace-b707-4fd9-8d47-8b60564e9a7e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:14.963904 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-51e32ace-b707-4fd9-8d47-8b60564e9a7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c84aa2376c247e197daa4eaf6a093ab', 'request_id': 'req-51e32ace-b707-4fd9-8d47-8b60564e9a7e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:14.970112 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-51e32ace-b707-4fd9-8d47-8b60564e9a7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c84aa2376c247e197daa4eaf6a093ab', 'request_id': 'req-51e32ace-b707-4fd9-8d47-8b60564e9a7e'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:15.054118 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-51e32ace-b707-4fd9-8d47-8b60564e9a7e admin admin] The token cache is being invalidate because project 4c84aa2376c247e197daa4eaf6a093ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 20:21:15.054418 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-51e32ace-b707-4fd9-8d47-8b60564e9a7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:15.055081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-51e32ace-b707-4fd9-8d47-8b60564e9a7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:15.056600 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1484/2977] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 20:21:14 2022] DELETE /identity/v3/projects/4c84aa2376c247e197daa4eaf6a093ab => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:21:15.061415 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d9d54-0690-4b2e-8111-d70859a3856a None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:15.063914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d9d54-0690-4b2e-8111-d70859a3856a None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:15.063914 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d9d54-0690-4b2e-8111-d70859a3856a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:15.081820 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-a07d9d54-0690-4b2e-8111-d70859a3856a None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:21:15.083043 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-a07d9d54-0690-4b2e-8111-d70859a3856a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:21:15.101421 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:21:15.142240 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1494/2978] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:21:15.150683 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa9d90-0258-4a2f-b734-9a4002cd0e91 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:15.151039 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa9d90-0258-4a2f-b734-9a4002cd0e91 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:15.151325 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa9d90-0258-4a2f-b734-9a4002cd0e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:15.172081 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-3baa9d90-0258-4a2f-b734-9a4002cd0e91 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:21:15.173298 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-3baa9d90-0258-4a2f-b734-9a4002cd0e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:21:15.181523 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1485/2979] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:21:15.185705 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d9d54-0690-4b2e-8111-d70859a3856a None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:21:15.195129 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d9d54-0690-4b2e-8111-d70859a3856a None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:21:15.199809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946aba9-f72f-44b9-a848-3e2b03cf7d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:21:15.200445 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946aba9-f72f-44b9-a848-3e2b03cf7d44 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:15.200727 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946aba9-f72f-44b9-a848-3e2b03cf7d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:15.201002 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946aba9-f72f-44b9-a848-3e2b03cf7d44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:15.201651 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946aba9-f72f-44b9-a848-3e2b03cf7d44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81868) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 20:21:15.201958 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946aba9-f72f-44b9-a848-3e2b03cf7d44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:21:15.203589 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946aba9-f72f-44b9-a848-3e2b03cf7d44 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:21:15.216596 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1495/2980] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 31 20:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 20:21:15.220971 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa9d90-0258-4a2f-b734-9a4002cd0e91 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:21:15.231080 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa9d90-0258-4a2f-b734-9a4002cd0e91 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:21:15.235737 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566b82c-95d0-43cb-bb69-d232e588898f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:21:15.236365 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566b82c-95d0-43cb-bb69-d232e588898f admin admin] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:15.236656 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566b82c-95d0-43cb-bb69-d232e588898f admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:15.236934 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566b82c-95d0-43cb-bb69-d232e588898f admin admin] PATH_INFO: `/v3/projects` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:15.237594 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566b82c-95d0-43cb-bb69-d232e588898f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:21:15.239568 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566b82c-95d0-43cb-bb69-d232e588898f admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:21:15.335673 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1486/2981] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 31 20:21:15 2022] POST /identity/v3/projects => generated 399 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:21:15.340096 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946aba9-f72f-44b9-a848-3e2b03cf7d44 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:21:15.351180 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946aba9-f72f-44b9-a848-3e2b03cf7d44 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:21:15.409159 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493bb87-8391-4c3e-9346-0b6fbe58166c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:21:15.409762 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493bb87-8391-4c3e-9346-0b6fbe58166c admin admin] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:15.410041 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493bb87-8391-4c3e-9346-0b6fbe58166c admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:15.410312 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493bb87-8391-4c3e-9346-0b6fbe58166c admin admin] PATH_INFO: `/v3/users` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:15.410969 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493bb87-8391-4c3e-9346-0b6fbe58166c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:21:15.413567 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493bb87-8391-4c3e-9346-0b6fbe58166c admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:21:15.516538 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1496/2982] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 31 20:21:15 2022] POST /identity/v3/users => generated 300 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 20:21:15.520628 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566b82c-95d0-43cb-bb69-d232e588898f admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:21:15.532297 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566b82c-95d0-43cb-bb69-d232e588898f admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:21:15.537303 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03396de6-5f19-42f5-a9d2-b964e3a2e971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:21:15.538097 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03396de6-5f19-42f5-a9d2-b964e3a2e971 admin admin] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:15.538382 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03396de6-5f19-42f5-a9d2-b964e3a2e971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:15.538660 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03396de6-5f19-42f5-a9d2-b964e3a2e971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:15.541111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03396de6-5f19-42f5-a9d2-b964e3a2e971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:21:15.541111 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03396de6-5f19-42f5-a9d2-b964e3a2e971 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:21:15.551601 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1487/2983] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:21:15.556012 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493bb87-8391-4c3e-9346-0b6fbe58166c admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:21:15.566086 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493bb87-8391-4c3e-9346-0b6fbe58166c admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:21:15.571962 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fcaeb-c266-42a6-8f1d-7c52ab228d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:21:15.572564 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fcaeb-c266-42a6-8f1d-7c52ab228d81 admin admin] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:15.572838 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fcaeb-c266-42a6-8f1d-7c52ab228d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:15.573107 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fcaeb-c266-42a6-8f1d-7c52ab228d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:15.573744 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40fcaeb-c266-42a6-8f1d-7c52ab228d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:21:15.575362 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40fcaeb-c266-42a6-8f1d-7c52ab228d81 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:21:15.587875 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1497/2984] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 31 20:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 20:21:15.592120 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03396de6-5f19-42f5-a9d2-b964e3a2e971 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:21:15.602625 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03396de6-5f19-42f5-a9d2-b964e3a2e971 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:21:15.607432 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee90817-157f-4b29-9abf-ed44f47604b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:21:15.608130 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee90817-157f-4b29-9abf-ed44f47604b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:15.608410 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee90817-157f-4b29-9abf-ed44f47604b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:15.608706 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee90817-157f-4b29-9abf-ed44f47604b6 admin admin] PATH_INFO: `/v3/projects/51ff92c1088040c787896ad4fc632599/users/a43e27d3852a4ecf9fb6873f126dd396/roles/357fdcf3417c45319b9d424a5ab4e9ca` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:15.623617 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee90817-157f-4b29-9abf-ed44f47604b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51ff92c1088040c787896ad4fc632599, user_id=a43e27d3852a4ecf9fb6873f126dd396, role_id=357fdcf3417c45319b9d424a5ab4e9ca)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:21:15.626244 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee90817-157f-4b29-9abf-ed44f47604b6 admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:21:15.728649 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1488/2985] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 31 20:21:15 2022] PUT /identity/v3/projects/51ff92c1088040c787896ad4fc632599/users/a43e27d3852a4ecf9fb6873f126dd396/roles/357fdcf3417c45319b9d424a5ab4e9ca => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:21:15.739248 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20caec9-e0b4-46d4-82df-c068bcfcbd7f None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:15.739534 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20caec9-e0b4-46d4-82df-c068bcfcbd7f None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:15.739805 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20caec9-e0b4-46d4-82df-c068bcfcbd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:15.767991 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-c20caec9-e0b4-46d4-82df-c068bcfcbd7f None None] MFA Rules not processed for user `a43e27d3852a4ecf9fb6873f126dd396`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:21:15.769108 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c20caec9-e0b4-46d4-82df-c068bcfcbd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:21:15.835008 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1498/2986] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:21:15 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:21:15.846486 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee90817-157f-4b29-9abf-ed44f47604b6 admin admin] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:21:15.848286 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-7ee90817-157f-4b29-9abf-ed44f47604b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:21:15.871339 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:21:15.908867 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee90817-157f-4b29-9abf-ed44f47604b6 admin admin] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:21:15.912612 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab29e84-9a95-4ffa-bad6-b9740c8682af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108a698aa3e6477b9ae896d68f22bd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:21:15.913214 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab29e84-9a95-4ffa-bad6-b9740c8682af service cinder] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:15.913488 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab29e84-9a95-4ffa-bad6-b9740c8682af service cinder] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:15.913758 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab29e84-9a95-4ffa-bad6-b9740c8682af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:15.922285 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab29e84-9a95-4ffa-bad6-b9740c8682af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:21:15.923747 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab29e84-9a95-4ffa-bad6-b9740c8682af service cinder] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:21:15.933495 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1489/2987] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:21:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:21:25.831270 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:21:26.086077 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=17 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:21:27.071402 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20caec9-e0b4-46d4-82df-c068bcfcbd7f None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:21:27.132827 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20caec9-e0b4-46d4-82df-c068bcfcbd7f None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:21:27.137321 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5132ce5-7a03-4489-9e4f-4f5064393021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:21:27.137940 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5132ce5-7a03-4489-9e4f-4f5064393021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:27.138219 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5132ce5-7a03-4489-9e4f-4f5064393021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:27.138492 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5132ce5-7a03-4489-9e4f-4f5064393021 admin admin] PATH_INFO: `/v3/users/a43e27d3852a4ecf9fb6873f126dd396` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:27.139642 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5132ce5-7a03-4489-9e4f-4f5064393021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a43e27d3852a4ecf9fb6873f126dd396)` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:21:27.141542 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5132ce5-7a03-4489-9e4f-4f5064393021 admin admin] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:21:27.339890 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c5132ce5-7a03-4489-9e4f-4f5064393021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a43e27d3852a4ecf9fb6873f126dd396'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:27.352161 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c5132ce5-7a03-4489-9e4f-4f5064393021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a43e27d3852a4ecf9fb6873f126dd396'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:27.430788 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c5132ce5-7a03-4489-9e4f-4f5064393021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a43e27d3852a4ecf9fb6873f126dd396'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:27.438535 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c5132ce5-7a03-4489-9e4f-4f5064393021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a43e27d3852a4ecf9fb6873f126dd396'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:27.439213 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.notifications [None req-c5132ce5-7a03-4489-9e4f-4f5064393021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a43e27d3852a4ecf9fb6873f126dd396'} {{(pid=81868) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:27.445902 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1499/2988] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 31 20:21:27 2022] DELETE /identity/v3/users/a43e27d3852a4ecf9fb6873f126dd396 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:21:27.450384 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e1c7a-2670-435a-aad5-7008625fc283 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:27.450845 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e1c7a-2670-435a-aad5-7008625fc283 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:27.450958 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e1c7a-2670-435a-aad5-7008625fc283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:27.469708 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-3d8e1c7a-2670-435a-aad5-7008625fc283 None None] MFA Rules not processed for user `30d09a6b4f0047b4ba7decc0e37bf0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:21:27.471023 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-3d8e1c7a-2670-435a-aad5-7008625fc283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:21:27.530329 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1490/2989] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 31 20:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:21:27.540628 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5132ce5-7a03-4489-9e4f-4f5064393021 admin admin] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:21:27.542375 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-c5132ce5-7a03-4489-9e4f-4f5064393021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:21:27.599669 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5132ce5-7a03-4489-9e4f-4f5064393021 admin admin] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:21:27.603825 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ddd94-1455-4cfe-94e7-7cd39fa852ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7a51545146846cc8284b7044b32198a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:21:27.604486 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ddd94-1455-4cfe-94e7-7cd39fa852ed service neutron] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:27.604764 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ddd94-1455-4cfe-94e7-7cd39fa852ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:27.605033 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ddd94-1455-4cfe-94e7-7cd39fa852ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:27.613992 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ddd94-1455-4cfe-94e7-7cd39fa852ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:21:27.615442 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ddd94-1455-4cfe-94e7-7cd39fa852ed service neutron] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:21:27.625809 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1500/2990] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:21:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:21:28.050101 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e1c7a-2670-435a-aad5-7008625fc283 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:21:28.061345 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-3d8e1c7a-2670-435a-aad5-7008625fc283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:21:28.066126 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e1c7a-2670-435a-aad5-7008625fc283 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:21:28.071672 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58766560-1f7a-4742-9604-08f7a040027c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d09a6b4f0047b4ba7decc0e37bf0cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9e5945a1a4d41fca11373bc396c02fd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:21:28.072527 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58766560-1f7a-4742-9604-08f7a040027c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:28.072813 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58766560-1f7a-4742-9604-08f7a040027c admin admin] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:28.073091 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58766560-1f7a-4742-9604-08f7a040027c admin admin] PATH_INFO: `/v3/projects/51ff92c1088040c787896ad4fc632599` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:28.074249 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58766560-1f7a-4742-9604-08f7a040027c admin admin] RBAC: Authorizing `identity:delete_project(project_id=51ff92c1088040c787896ad4fc632599)` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:21:28.076251 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58766560-1f7a-4742-9604-08f7a040027c admin admin] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:21:28.208638 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-58766560-1f7a-4742-9604-08f7a040027c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51ff92c1088040c787896ad4fc632599', 'request_id': 'req-58766560-1f7a-4742-9604-08f7a040027c'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:28.214894 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-58766560-1f7a-4742-9604-08f7a040027c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51ff92c1088040c787896ad4fc632599', 'request_id': 'req-58766560-1f7a-4742-9604-08f7a040027c'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:28.221071 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-58766560-1f7a-4742-9604-08f7a040027c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51ff92c1088040c787896ad4fc632599', 'request_id': 'req-58766560-1f7a-4742-9604-08f7a040027c'} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:28.313586 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-58766560-1f7a-4742-9604-08f7a040027c admin admin] The token cache is being invalidate because project 51ff92c1088040c787896ad4fc632599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81869) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 20:21:28.313883 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-58766560-1f7a-4742-9604-08f7a040027c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:28.314554 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.notifications [None req-58766560-1f7a-4742-9604-08f7a040027c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81869) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 20:21:28.316106 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1491/2991] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 31 20:21:28 2022] DELETE /identity/v3/projects/51ff92c1088040c787896ad4fc632599 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 20:21:37.621339 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:21:38.310299 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:21:43.406260 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add5142-fc67-45f9-a169-38ab5fbb3ee9 None None] REQUEST_METHOD: `POST` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:43.406832 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add5142-fc67-45f9-a169-38ab5fbb3ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:43.406832 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add5142-fc67-45f9-a169-38ab5fbb3ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:43.427502 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.auth.core [None req-1add5142-fc67-45f9-a169-38ab5fbb3ee9 None None] MFA Rules not processed for user `eced13cc4030423abe63bb1527f6d35a`. Rule list: `[]` (Enabled: `True`). {{(pid=81868) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:21:43.428706 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.fernet_utils [None req-1add5142-fc67-45f9-a169-38ab5fbb3ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81868) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:21:43.490820 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1501/2992] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Aug 31 20:21:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:21:43.504119 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc77555-7e52-4681-821e-bf46121356d8 None None] REQUEST_METHOD: `POST` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:43.504566 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc77555-7e52-4681-821e-bf46121356d8 None None] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:43.504670 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc77555-7e52-4681-821e-bf46121356d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:43.524318 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.auth.core [None req-bdc77555-7e52-4681-821e-bf46121356d8 None None] MFA Rules not processed for user `eced13cc4030423abe63bb1527f6d35a`. Rule list: `[]` (Enabled: `True`). {{(pid=81869) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 20:21:43.525575 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.fernet_utils [None req-bdc77555-7e52-4681-821e-bf46121356d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81869) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 20:21:43.534754 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1492/2993] 10.222.0.59 () {58 vars in 1058 bytes} [Wed Aug 31 20:21:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 20:21:43.540597 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add5142-fc67-45f9-a169-38ab5fbb3ee9 None None] Authenticating user token {{(pid=81868) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:21:43.551818 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add5142-fc67-45f9-a169-38ab5fbb3ee9 None None] Validating token access rules against request {{(pid=81868) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:21:43.556808 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20cc24-f8d4-429c-919d-a6e61d81b045 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81868) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:21:43.557402 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20cc24-f8d4-429c-919d-a6e61d81b045 service placement] REQUEST_METHOD: `GET` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:21:43.557683 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20cc24-f8d4-429c-919d-a6e61d81b045 service placement] SCRIPT_NAME: `/identity` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:21:43.557956 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20cc24-f8d4-429c-919d-a6e61d81b045 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81868) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:21:43.566487 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20cc24-f8d4-429c-919d-a6e61d81b045 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:21:43.568094 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20cc24-f8d4-429c-919d-a6e61d81b045 service placement] RBAC: Authorization granted {{(pid=81868) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:21:43.577798 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: [pid: 81868|app: 0|req: 1502/2994] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:21:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:21:53.519465 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=3 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:21:53.573570 dv-cinder-852209-9-iscsi devstack@keystone.service[81868]: DEBUG dbcounter [-] [81868] Writing DB stats keystone:SELECT=35 {{(pid=81868) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 20:26:45.362978 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc77555-7e52-4681-821e-bf46121356d8 None None] Authenticating user token {{(pid=81869) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 20:26:45.423402 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc77555-7e52-4681-821e-bf46121356d8 None None] Validating token access rules against request {{(pid=81869) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 20:26:45.428596 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae6ba8-e039-4730-8f97-d4ca5467cebc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eced13cc4030423abe63bb1527f6d35a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '161c90d34bb34925a1aaeb70c7de7eae', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81869) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 20:26:45.429210 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae6ba8-e039-4730-8f97-d4ca5467cebc service placement] REQUEST_METHOD: `GET` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 20:26:45.429495 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae6ba8-e039-4730-8f97-d4ca5467cebc service placement] SCRIPT_NAME: `/identity` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 20:26:45.429780 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae6ba8-e039-4730-8f97-d4ca5467cebc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81869) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 20:26:45.438761 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae6ba8-e039-4730-8f97-d4ca5467cebc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 20:26:45.440358 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae6ba8-e039-4730-8f97-d4ca5467cebc service placement] RBAC: Authorization granted {{(pid=81869) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 20:26:45.449859 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: [pid: 81869|app: 0|req: 1493/2995] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 31 20:26:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 20:26:55.445975 dv-cinder-852209-9-iscsi devstack@keystone.service[81869]: DEBUG dbcounter [-] [81869] Writing DB stats keystone:SELECT=27 {{(pid=81869) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}